Download LSF Parallel User's Guide

LSF Parallel User’s Guide
/6)9HUVLRQ
)LUVW(GLWLRQ$XJXVW
Platform Computing Corporation
ii
LSF Parallel User’s Guide
/6)3DUDOOHO8VHU¶V*XLGH
Copyright © 1998 Platform Computing Corporation
All rights reserved.
This document is copyrighted. This document may not, in whole or part, be copied, duplicated,
reproduced, translated, electronically stored, or reduced to machine readable form without prior
written consent from Platform Computing Corporation.
Although the material contained herein has been carefully reviewed, Platform Computing
Corporation does not warrant it to be free of errors or omissions. Platform Computing
Corporation reserves the right to make corrections, updates, revisions or changes to the
information contained herein.
UNLESS PROVIDED OTHERWISE IN WRITING BY PLATFORM COMPUTING
CORPORATION, THE PROGRAM DESCRIBED HEREIN IS PROVIDED AS IS WITHOUT
WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE. IN NO EVENT WILL PLATFORM BE LIABLE TO ANYONE FOR
SPECIAL, COLLATERAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
ANY LOST PROFITS OR LOST SAVINGS, ARISING OUT OF THE USE OF OR INABILITY TO
USE THIS PROGRAM.
LSF, LSF Base, LSF Batch, LSF JobScheduler, LSF MultiCluster, LSF Make, LSF Analyzer, LSF
Parallel, Platform Computing, and the Platform Computing and LSF logos are trademarks of
Platform Computing Corporation.
Other products or services mentioned in this document are identified by the trademarks or
service marks of their respective companies or organizations.
Printed in Canada.
/6)3DUDOOHO8VHU·V*XLGH
iii
Revision Information for the LSF Parallel User’s Guide
iv
(GLWLRQ
'HVFULSWLRQ
First
This document describes the LSF Parallel system released with LSF Suite
version 3.2.
/6)3DUDOOHO8VHU·V*XLGH
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
$XGLHQFHYLLL
5HODWHG3XEOLFDWLRQVL[
7\SRJUDSKLF&RQYHQWLRQV [L
&RPPDQG1RWDWLRQ[LL
/6)6XLWH [LLL
7HFKQLFDO$VVLVWDQFH [Y
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
:KDW,VWKH/6)3DUDOOHO6\VWHP"
+RZ'RHV/6)3DUDOOHO)LW,QWRWKH/6)%DWFK6\VWHP" /6)3DUDOOHO$UFKLWHFWXUH
2. Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
:ULWLQJD'LVWULEXWHG$SSOLFDWLRQ
&RPSLOLQJDQG/LQNLQJWKH$SSOLFDWLRQ
5XQQLQJWKH$SSOLFDWLRQ
3. Building Parallel Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 13
,QFOXGLQJWKH+HDGHU)LOH &RPSLOLQJDQG/LQNLQJ %XLOGLQJD+HWHURJHQHRXV3DUDOOHO$SSOLFDWLRQ 4. Submitting Parallel Applications . . . . . . . . . . . . . . . . . . . . . . . . 19
-RE6XEPLVVLRQ0HWKRGV
%DWFK([HFXWLRQ
%DWFK-RE6WDWXV 6XEPLWWLQJ%DWFK-REV 6XVSHQGLQJ-REV 5HVXPLQJ-REV
0RQLWRULQJ-RE6WDWXV
7HUPLQDWLQJ-REV
5XQQLQJ+HWHURJHQHRXV3DUDOOHO$SSOLFDWLRQV /6)3DUDOOHO8VHU·V*XLGH
v
,QWHUDFWLYH([HFXWLRQ
7KHSDP&RPPDQG
3URFHVV6WDWXV5HSRUW *HWWLQJ+RVW,QIRUPDWLRQ A. Vendor MPI Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . 41
+303, 6*,03, 681+3&03, Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
vi
/6)3DUDOOHO8VHU·V*XLGH
Preface
7KH/6)3DUDOOHO8VHU
V*XLGHGHVFULEHVKRZWRFRPSLOHDQGOLQNH[HFXWHLQWHUDFW
DQGPRQLWRUSDUDOOHODSSOLFDWLRQVVXEPLWWHGWKURXJKWKH/6)6XLWHRISURGXFWV
)RUWKHPRVWSDUWWKLVJXLGHGRHVQRWUHSHDWLQIRUPDWLRQWKDWLVDYDLODEOHLQGHWDLO
HOVHZKHUHEXWIRFXVHVRQZKDWLVVSHFLILFWRXVLQJWKH3ODWIRUP&RPSXWLQJ
&RUSRUDWLRQ/6)3DUDOOHOV\VWHP5HIHUHQFHVWRPRUHJHQHUDOVRXUFHVDUHSURYLGHG
LQ5HODWHG3XEOLFDWLRQVRQSDJHL[LQWKLVSUHIDFH
7KLVSUHIDFHGLVFXVVHVWKHIROORZLQJWRSLFV
$XGLHQFHYLLL
5HODWHG3XEOLFDWLRQV L[
5HODWHG3XEOLFDWLRQV L[
7\SRJUDSKLF&RQYHQWLRQV [L
&RPPDQG1RWDWLRQ [LL
/6)6XLWH[LLL
7HFKQLFDO$VVLVWDQFH[Y
LSF Parallel User’s Guide vii
Preface
Audience
Audience
7KLVJXLGHSURYLGHVUHIHUHQFHDQGWXWRULDOPDWHULDOIRU
‡
03,SURJUDPPHUVZKRZDQWWRFRPSLOHDQGOLQN03,SURJUDPVIRUXVHZLWK
WKH/6)3DUDOOHOV\VWHP
‡ 8VHUVRIWKH/6)3DUDOOHOV\VWHPZKRZDQWWRVXEPLWH[HFXWHPRQLWRUDQG
LQWHUDFWZLWKSDUDOOHODSSOLFDWLRQVXVLQJWKH/6)%DWFKV\VWHP
7KHXVHUVRIWKLVJXLGHDUHH[SHFWHGWREHIDPLOLDUZLWK
‡
‡
‡
3URJUDPPLQJLQWKH&RU)RUWUDQODQJXDJH
0HVVDJH3DVVLQJ,QWHUIDFH03,FRQFHSWV
7KH/6)%DWFKV\VWHP
viii LSF Parallel User’s Guide
Preface
Related Publications
Related Publications
7KLVJXLGHIRFXVHVRQXVLQJSDUDOOHODSSOLFDWLRQVZLWKWKH/6)6XLWHRISURGXFWV
SULPDULO\WKH/6)%DWFKV\VWHP,WDVVXPHVIDPLOLDULW\ZLWKWKH/6)6XLWHRI
SURGXFWVDQGWKH03,VWDQGDUG7KHIROORZLQJPDWHULDOVSURYLGHXVHIXOEDFNJURXQG
DERXWXVLQJWKH/6)6XLWHRISURGXFWVDQG03,
These guides are available from Platform Computing Corporation:
‡
‡
‡
‡
‡
‡
‡
‡
‡
/6)6XLWH,QVWDOODWLRQ*XLGH
/6)%DWFK$GPLQLVWUDWRU·V*XLGH
/6)%DWFK$GPLQLVWUDWRU·V4XLFN5HIHUHQFH
/6)%DWFK8VHU·V*XLGH
/6)%DWFK8VHU·V4XLFN5HIHUHQFH
/6)-RE6FKHGXOHU$GPLQLVWUDWRU·V*XLGH
/6)-RE6FKHGXOHU8VHU
V*XLGH
/6)$QDO\]HU8VHU
V*XLGH
/6)3URJUDPPHU·V*XLGH
This document is available on the World Wide Web:
03,$0HVVDJH3DVVLQJ,QWHUIDFH6WDQGDUG0HVVDJH3DVVLQJ,QWHUIDFH)RUXP
8QLYHUVLW\RI7HQQHVVHH
http://www.mcs.anl.gov/mpi/mpi-report-1.1/mpi-report.html
These books should be available at your local bookstore:
03,7KH&RPSOHWH5HIHUHQFHE\0DUF6QLU6WHYH:2WWR6WHYHQ+XVV/HGHUPDQ
'DYLG::DONHUDQG-DFN'RQJDUUD0,73UHVV
8VLQJ03,E\:LOOLDP*URSS(ZLQJ/XVNDQG$QWKRQ\6NMHOOXP0,73UHVV
3DUDOOHO3URJUDPPLQJZLWK03,E\3HWHU3DFKHFR0RUJDQ.DXIPDQQ3XEOLVKHUV,QF
'HVLJQLQJDQG%XLOGLQJ3DUDOOHO3URJUDPV,DQ)RVWHU$GGLVRQ:HVOH\
LSF Parallel User’s Guide
ix
Preface
Related Publications
Online documentation supplied by Platform Computing Corporation:
0DQSDJHVDFFHVVHGZLWKWKHmanFRPPDQGIRUDOOFRPPDQGVDQG03,IXQFWLRQV
x
LSF Parallel User’s Guide
Preface
Typographic Conventions
Typographic Conventions
7DEOH7KHW\SRJUDSKLFFRQYHQWLRQVXVHGLQWKLVJXLGH
7\SHIDFH
0HDQLQJ
AaBbCcDdEeFfGg 7KHQDPHVRIFRPPDQGVILOHVDQGGLUHFWRULHVRQVFUHHQ
FRPSXWHURXWSXW
AaBbCcDdEeFfGg
:KDW\RXW\SHFRQWUDVWHGZLWKRQVFUHHQFRPSXWHURXWSXW
AaBbCcDdEeFfGg &RPPDQGOLQHSODFHKROGHUUHSODFHZLWKUHDOQDPHRUYDOXH
$D%E&F'G(H)I*J %RRNWLWOHVQHZZRUGVRUWHUPVRUZRUGVWREHHPSKDVL]HG
LSF Parallel User’s Guide
xi
Preface
Command Notation
Command Notation
7DEOH7KHFRPPDQGQRWDWLRQXVHGLQWKLVJXLGH
1RWDWLRQ
0HDQLQJ
$SRVWURSKHVRU
0XVWEHHQWHUHGH[DFWO\DVVKRZQ
&RPPDV
0XVWEHHQWHUHGH[DFWO\DVVKRZQ
(OOLSVLV«
7KHSUHFHGLQJSDUDPHWHUFDQEHUHSHDWHG'RQRWHQWHUWKH
HOOLSVLV
ORZHUFDVHLWDOLFV
7KHSDUDPHWHUPXVWEHVXEVWLWXWHG
25EDU_
<RXPXVWHQWHURQHRIWKHLWHPVVHSDUDWHGE\WKHEDU<RX
FDQQRWHQWHUPRUHWKDQRQHLWHP
3DUHQWKHVLV
0XVWEHHQWHUHGH[DFWO\DVVKRZQ
3DUDPHWHULQ
7KHSDUDPHWHUVZLWKLQWKHEUDFNHWVLVRSWLRQDO'RQRWHQWHU
VTXDUHEUDFNHWV>@ WKHEUDFNHWV
6WDFNHGEUDFHV
<RXPXVWHQWHURQHRIWKHLWHPVZLWKLQWKHEUDFHV<RX
^`
FDQQRWHQWHUPRUHWKDQRQHLWHP'RQRWHQWHUWKHEUDFHV
^`
&VKHOOSURPSW 8QOHVVRWKHUZLVHQRWHGWKH&VKHOOSURPSWLVXVHGLQDOO
FRPPDQGH[DPSOHV
xii LSF Parallel User’s Guide
Preface
LSF Suite 3.2
LSF Suite 3.2
/6)LVDVXLWHRIZRUNORDGPDQDJHPHQWSURGXFWVLQFOXGLQJWKHIROORZLQJ
/6)%DWFK
LVDEDWFKMRESURFHVVLQJV\VWHPIRUGLVWULEXWHGDQG
KHWHURJHQHRXVHQYLURQPHQWVZKLFKHQVXUHVRSWLPDO
UHVRXUFHVKDULQJ
/6)-RE6FKHGXOHU
LVDGLVWULEXWHGSURGXFWLRQMREVFKHGXOLQJDSSOLFDWLRQWKDW
LQWHJUDWHVKHWHURJHQHRXVVHUYHUVLQWRDYLUWXDOPDLQIUDPH
RUYLUWXDOVXSHUFRPSXWHU
/6)0XOWL&OXVWHU
VXSSRUWVUHVRXUFHVKDULQJDPRQJPXOWLSOHFOXVWHUVRI
FRPSXWHUVXVLQJ/6)SURGXFWVZKLOHPDLQWDLQLQJUHVRXUFH
RZQHUVKLSDQGFOXVWHUDXWRQRP\
/6)$QDO\]HU
LVDJUDSKLFDOWRROIRUFRPSUHKHQVLYHZRUNORDGGDWD
DQDO\VLV,WSURFHVVHVFOXVWHUZLGHMREORJVIURP/6)%DWFK
DQG/6)-RE6FKHGXOHUWRSURGXFHVWDWLVWLFDOUHSRUWVRQWKH
XVDJHRIV\VWHPUHVRXUFHVE\XVHUVRQGLIIHUHQWKRVWV
WKURXJKYDULRXVTXHXHV
/6)3DUDOOHO
LVDVRIWZDUHSURGXFWWKDWPDQDJHVSDUDOOHOMREH[HFXWLRQLQ
DSURGXFWLRQQHWZRUNHQYLURQPHQW
/6)0DNH
LVDGLVWULEXWHGDQGSDUDOOHO0DNHEDVHGRQ*180DNHWKDW
VLPXOWDQHRXVO\GLVSDWFKHVWDVNVWRPXOWLSOHKRVWV
/6)%DVH
LVWKHVRIWZDUHXSRQZKLFKDOOWKHRWKHU/6)SURGXFWVDUH
EDVHG,WLQFOXGHVWKHQHWZRUNVHUYHUV/,0DQG5(6WKH
/6)$3,DQGORDGVKDULQJWRROV
7KHUHDUHWZRHGLWLRQVRIWKH/6)6XLWH
LSF Enterprise Edition
3ODWIRUP
V/6)(QWHUSULVH(GLWLRQSURYLGHVDUHOLDEOHVFDODEOHPHDQVIRU
RUJDQL]DWLRQVWRVFKHGXOHDQDO\]HDQGPRQLWRUWKHLUGLVWULEXWHGZRUNORDGVDFURVV
LSF Parallel User’s Guide xiii
Preface
LSF Suite 3.2
KHWHURJHQHRXV81,;DQG:LQGRZV17FRPSXWLQJHQYLURQPHQWV/6)(QWHUSULVH
(GLWLRQLQFOXGHVDOOWKHIHDWXUHVLQ/6)6WDQGDUG(GLWLRQ/6)%DVHDQG/6)%DWFK
SOXVWKHEHQHILWVRI/6)$QDO\]HUDQG/6)0XOWL&OXVWHU
LSF Standard Edition
7KHIRXQGDWLRQIRUDOO/6)SURGXFWV3ODWIRUP
V6WDQGDUG(GLWLRQFRQVLVWVRIWZR
SURGXFWV/6)%DVHDQG/6)%DWFK/6)6WDQGDUG(GLWLRQRIIHUVXVHUVUREXVWORDG
VKDULQJDQGVRSKLVWLFDWHGEDWFKVFKHGXOLQJDFURVVGLVWULEXWHG81,;DQG:LQGRZV
17FRPSXWLQJHQYLURQPHQWV
xiv LSF Parallel User’s Guide
Preface
Technical Assistance
Technical Assistance
&RQWDFW\RXUUHVHOOHUIRU7HFKQLFDO$VVLVWDQFH<RXFDQDOVRFRQWDFWWKH7HFKQLFDO
6XSSRUW'HSDUWPHQWDW3ODWIRUP&RPSXWLQJ&RUSRUDWLRQLQWKHIROORZLQJZD\V
‡
0DLO
/6)7HFKQLFDO6XSSRUW
3ODWIRUP&RPSXWLQJ&RUSRUDWLRQ
WK$YHQXH
0DUNKDP2QWDULR
&DQDGD/57
‡
7HOHSKRQH
‡
7ROOIUHH
3/$7)250
‡
)D[
‡
(PDLO
VXSSRUW#SODWIRUPFRP
:KHQFRQWDFWLQJ3ODWIRUP&RPSXWLQJSOHDVHLQFOXGHWKHIXOOQDPHRI\RXU
FRPSDQ\
Note
$OWHUQDWLYHO\\RXFDQILQGDQVZHUVWRPDQ\RI\RXUTXHVWLRQVE\YLVLWLQJWKH
3ODWIRUP&RPSXWLQJ&RUSRUDWLRQKRPHSDJHRQWKH:RUOG:LGH:HE3RLQW\RXU
EURZVHUWR
http://www.platform.com
3ODWIRUPZHOFRPHV\RXUFRPPHQWVDQGVXJJHVWLRQVRQWKLVGRFXPHQWVHQGHPDLO
WRGRF#SODWIRUPFRPRUFRQWDFWXVE\PDLOID[RUSKRQH
LSF Parallel User’s Guide xv
Preface
Technical Assistance
xvi LSF Parallel User’s Guide
1 Introduction
7KLVFKDSWHUGHVFULEHVWKH/6)3DUDOOHOV\VWHPDQGLWVDUFKLWHFWXUH
7KLVFKDSWHUGLVFXVVHVWKHIROORZLQJWRSLFV
:KDW,VWKH/6)3DUDOOHO6\VWHP"
+RZ'RHV/6)3DUDOOHO)LW,QWRWKH/6)%DWFK6\VWHP" /6)3DUDOOHO$UFKLWHFWXUH
LSF Parallel User’s Guide
1
1
Introduction
What Is the LSF Parallel System?
What Is the LSF Parallel System?
7KH/6)3DUDOOHOV\VWHPLVDIXOO\VXSSRUWHGFRPPHUFLDOVRIWZDUHV\VWHPWKDW
VXSSRUWVWKHSURJUDPPLQJWHVWLQJDQGH[HFXWLRQRISDUDOOHODSSOLFDWLRQVLQ
SURGXFWLRQHQYLURQPHQWV
7KH/6)3DUDOOHOV\VWHPLVIXOO\LQWHJUDWHGZLWKWKH/6)%DWFKV\VWHPWKHGHIDFWR
LQGXVWU\VWDQGDUGUHVRXUFHPDQDJHPHQWVRIWZDUHSURGXFWWRSURYLGHORDGVKDULQJ
LQDGLVWULEXWHGV\VWHPDQGEDWFKVFKHGXOLQJIRUFRPSXWHLQWHQVLYHMREV7KH/6)
3DUDOOHOV\VWHPSURYLGHVVXSSRUWIRU
‡
‡
‡
‡
‡
‡
2
'\QDPLFUHVRXUFHGLVFRYHU\DQGDOORFDWLRQUHVRXUFHUHVHUYDWLRQIRUSDUDOOHO
EDWFKMREH[HFXWLRQ
7UDQVSDUHQWLQYRFDWLRQRIWKHGLVWULEXWHGMRESURFHVVHV
)XOOFRQWURORIWKHGLVWULEXWHGMRESURFHVVHVWRHQVXUHQRSURFHVVHVZLOO
EHFRPHXQPDQDJHG7KLVHIIHFWLYHO\UHGXFHVWKHSRVVLELOLW\RIRQHSDUDOOHOMRE
FDXVLQJVHYHUHGLVUXSWLRQWRDQRUJDQL]DWLRQ
VFRPSXWHUVHUYLFH
7KHVWDQGDUG03,LQWHUIDFH
$OOPDMRU81,;RSHUDWLQJV\VWHPV
)XOOLQWHJUDWLRQZLWKWKH/6)%DWFKV\VWHPSURYLGLQJKHWHURJHQHRXVUHVRXUFH
EDVHGEDWFKMREVFKHGXOLQJ
LSF Parallel User’s Guide
Introduction
How Does LSF Parallel Fit Into the LSF Batch System?
How Does LSF Parallel Fit Into the LSF Batch System?
7KH/6)3DUDOOHOV\VWHPDGRSWVDOD\HUHGDSSURDFKVKRZQLQ)LJXUHWKDWLVIXOO\
LQWHJUDWHGZLWKWKH/6)%DWFKV\VWHP,QDGGLWLRQWRWKH/6)%DWFKV\VWHP
UHVRXUFHVWKHIROORZLQJFRPSRQHQWVPDNHXSWKH/6)3DUDOOHOV\VWHP
‡
‡
7KH03,/LEUDU\
7KH3DUDOOHO$SSOLFDWLRQ0DQDJHU3$0
)LJXUH0DMRU&RPSRQHQWVRIWKH/6)3DUDOOHOV\VWHP
MPI Library
7KH0HVVDJH3DVVLQJ,QWHUIDFH03,OLEUDU\LVDPHVVDJHSDVVLQJOLEUDU\WKDWPXVW
EHOLQNHGWRWKHSDUDOOHODSSOLFDWLRQVWKDWDUHWREHUXQLQWKH/6)%DWFKV\VWHP
7KH03,OLEUDU\WUDQVODWHV03,PHVVDJHFDOOVWRPHVVDJHVIRUWKHPDFKLQH
GHSHQGHQWOD\HUDQGLWLQWHUIDFHVWKHXVHUDSSOLFDWLRQWR3$0
6HH9HQGRU03,,PSOHPHQWDWLRQVRQSDJHIRUDGHVFULSWLRQRIYHQGRUVSHFLILF03,
LPSOHPHQWDWLRQV
LSF Parallel User’s Guide
3
1
Introduction
How Does LSF Parallel Fit Into the LSF Batch System?
PAM
7KH3DUDOOHO$SSOLFDWLRQ0DQDJHU3$0LVWKHSRLQWRIFRQWUROIRUWKH/6)3DUDOOHO
V\VWHP3$0LVIXOO\LQWHJUDWHGZLWKWKH/6)%DWFKV\VWHP3$0LQWHUIDFHVWKH
XVHUDSSOLFDWLRQZLWKWKH/6)%DWFKV\VWHP)RUDOOSDUDOOHODSSOLFDWLRQSURFHVVHV
WDVNV3$0
‡
‡
‡
‡
‡
‡
‡
0DLQWDLQVWKHFRPPXQLFDWLRQFRQQHFWLRQPDS
0RQLWRUVDQGIRUZDUGVFRQWUROVLJQDOV
5HFHLYHVUHTXHVWVWRDGGGHOHWHVWDUWDQGFRQQHFWWDVNV
0RQLWRUVUHVRXUFHXVDJHZKLOHWKHXVHUDSSOLFDWLRQLVUXQQLQJ
(QIRUFHVMREOHYHOUHVRXUFHOLPLWV
&ROOHFWVUHVRXUFHXVDJHLQIRUPDWLRQDQGH[LWVWDWXVXSRQWHUPLQDWLRQ
+DQGOHVVWDQGDUG,2
LSF Batch System
7KH/6)%DWFKV\VWHPLVDVRSKLVWLFDWHGUHVRXUFHEDVHGEDWFKMREVFKHGXOLQJ
V\VWHP,WDFFHSWVXVHUMREVDQGKROGVWKHPLQTXHXHVXQWLOVXLWDEOHKRVWVDUH
DYDLODEOHDQGUHVRXUFHUHTXLUHPHQWVDUHVDWLVILHG+RVWVHOHFWLRQLVEDVHGRQXSWR
WKHPLQXWHORDGLQIRUPDWLRQSURYLGHGE\WKHPDVWHU/RDG,QIRUPDWLRQ0DQDJHU
/,0
/6)%DWFKUXQVXVHUMREVRQEDWFKVHUYHUKRVWV,WKDVVRSKLVWLFDWHGFRQWUROVIRU
VKDULQJKRVWVZLWKLQWHUDFWLYHXVHUVWKHUHLVQRQHHGWRVHWDVLGHGHGLFDWHGKRVWV
IRUSURFHVVLQJEDWFKMREV
6HHWKH/6)%DWFK8VHU·V*XLGHDQGWKH/6)%DWFK$GPLQLVWUDWRU·V*XLGHIRUDGHWDLOHG
GHVFULSWLRQRIWKH/6)%DWFKV\VWHP
4
LSF Parallel User’s Guide
Introduction
LSF Parallel Architecture
LSF Parallel Architecture
7KH/6)3DUDOOHOV\VWHPIXOO\XWLOL]HVWKHUHVRXUFHVRIWKH/6)%DWFK6\VWHPIRU
UHVRXUFHVHOHFWLRQDQGSURFHVVLQYRFDWLRQDQGFRQWURO7KHSURFHVVRISDUDOOHOEDWFK
MRELQYRFDWLRQDQGFRQWUROLVVKRZQLQ)LJXUHDQGGHVFULEHGLQ7DEOHRQSDJH
7KH/6)FRPSRQHQWVVKRZQLQ)LJXUHDUHGHVFULEHGLQ7DEOHRQSDJH
)LJXUH/6)3DUDOOHO$UFKLWHFWXUH
7KH/6)3DUDOOHOV\VWHPDOVRVXSSRUWVLQWHUDFWLYHSDUDOOHOMREVXEPLVVLRQ7KH
SURFHVVLVVLPLODUWRWKDWVKRZQLQ)LJXUHH[FHSWWKHXVHUUHTXHVWLVVXEPLWWHG
GLUHFWO\WR3$0ZKLFKPDNHVDVLPSOHSODFHPHQWTXHU\WR/,0-RETXHXLQJDQG
UHVRXUFHUHVHUYDWLRQVDUHQRWVXSSRUWHGLQLQWHUDFWLYHPRGH
LSF Parallel User’s Guide
5
1
Introduction
LSF Parallel Architecture
7DEOH/6)3DUDOOHO%DWFK-RE,QYRFDWLRQ
6WDJH
'HVFULSWLRQ
8VHUVXEPLWVDSDUDOOHOEDWFKMREWRWKH/6)%DWFKV\VWHP
0%'UHWULHYHVDOLVWRIVXLWDEOHH[HFXWLRQKRVWVIURPWKHPDVWHU/,0
0%'DOORFDWHVVFKHGXOHVUHVHUYHVWKHH[HFXWLRQKRVWVIRUWKHSDUDOOHO
EDWFKMRE
0%'GLVSDWFKHVWKHSDUDOOHOEDWFKMREWRWKH6%'RQWKHILUVWH[HFXWLRQ
KRVWWKDWZDVDOORFDWHGWRWKHEDWFKMRE
6%'VWDUWV3$0RQWKHVDPHH[HFXWLRQKRVW
3$0VWDUWV5(6RQHDFKH[HFXWLRQKRVWDOORFDWHGWRWKHEDWFKMRE
5(6VWDUWVWKHWDVNVRQHDFKH[HFXWLRQKRVW
7DEOH'HVFULSWLRQRI/6)3DUDOOHO&RPSRQHQWV
3DUW
8VHU5HTXHVW
0%'
/,0
)XQFWLRQ
%DWFKMREVXEPLVVLRQWRWKH/6)%DWFKV\VWHPXVLQJWKHbsub
FRPPDQG
7KH0DVWHU%DWFK'DHPRQLVWKHSROLF\FHQWHUIRUWKH/6)
%DWFKV\VWHP,WPDLQWDLQVLQIRUPDWLRQDERXWEDWFKMREV
KRVWVXVHUVDQGTXHXHV$OORIWKLVLQIRUPDWLRQLVXVHGLQ
VFKHGXOLQJEDWFKMREVWRKRVWV
7KH/RDG,QIRUPDWLRQ0DQDJHULVDGDHPRQSURFHVVUXQQLQJ
RQHDFKH[HFXWLRQKRVW/,0PRQLWRUVWKHORDGRQLWVKRVW
DQGH[FKDQJHVWKLVLQIRUPDWLRQZLWKWKH0DVWHU/,0
7KH0DVWHU/,0UHVLGHVRQRQHH[HFXWLRQKRVWDQGFROOHFWV
LQIRUPDWLRQIURPWKH/,0VRQDOORWKHUKRVWVLQWKH/6)
FOXVWHU,IWKHPDVWHU/,0EHFRPHVXQDYDLODEOHDQRWKHUKRVW
ZLOODXWRPDWLFDOO\WDNHRYHU
)RUEDWFKVXEPLVVLRQWKHPDVWHU/,0SURYLGHVWKLV
LQIRUPDWLRQLVSURYLGHGWRWKH0%'
)RULQWHUDFWLYHH[HFXWLRQWKH0DVWHU/,0SURYLGHVVLPSOH
SODFHPHQWDGYLFH
6
LSF Parallel User’s Guide
Introduction
LSF Parallel Architecture
7DEOH'HVFULSWLRQRI/6)3DUDOOHO&RPSRQHQWV
6%'
3$0
7KH6ODYH%DWFK'DHPRQVDUHEDWFKMREH[HFXWLRQDJHQWV
UHVLGLQJRQWKHH[HFXWLRQKRVWV6%'UHFHLYHVMREVIURPWKH
0%'LQWKHIRUPRIDMREVSHFLILFDWLRQDQGVWDUWV5(6WRUXQ
WKHMREDFFRUGLQJWKHVSHFLILFDWLRQ6%'UHSRUWVWKHEDWFK
MREVWDWXVWRWKH0%'ZKHQHYHUMREVWDWHFKDQJHV
7KH3DUDOOHO$SSOLFDWLRQ0DQDJHULVWKHSRLQWRIFRQWUROIRU
WKH/6)3DUDOOHOV\VWHP3$0LVIXOO\LQWHJUDWHGZLWKWKH/6)
6\VWHP3$0LQWHUIDFHVWKHXVHUDSSOLFDWLRQZLWKWKH/6)
V\VWHP
,I3$0RULWVKRVWFUDVKHVHDFK5(6ZLOOWHUPLQDWHDOOWDVNV
XQGHULWVPDQDJHPHQW7KLVDYRLGVWKHSUREOHPRIRUSKDQHG
SURFHVVHV
5(6
7KH5HPRWH([HFXWLRQ6HUYHUVUHVLGHRQHDFKH[HFXWLRQ
KRVW5(6PDQDJHVDOOUHPRWHWDVNVDQGIRUZDUGVVLJQDOV
VWDQGDUG,2UHVRXUFHVFRQVXPSWLRQGDWDDQGSDUDOOHOMRE
LQIRUPDWLRQEHWZHHQ3$0DQGWKHWDVNV
DSSOLFDWLRQWDVN
7KHLQGLYLGXDOSURFHVVRIDSDUDOOHODSSOLFDWLRQ
H[HFXWLRQKRVWV
7KHPRVWVXLWDEOHKRVWVWRH[HFXWHWKHEDWFKMREDV
GHWHUPLQHGE\WKH/6)%DWFKV\VWHP
ILUVWH[HFXWLRQKRVW 7KHKRVWQDPHDWWKHWRSRIWKHH[HFXWLRQKRVWOLVWDV
GHWHUPLQHGE\WKH/6)%DWFKV\VWHP
LSF Parallel User’s Guide
7
1
8
Introduction
LSF Parallel Architecture
LSF Parallel User’s Guide
Getting Started
2 Getting Started
7KHSXUSRVHRIWKLVFKDSWHULVWRTXLFNO\LQWURGXFHWKHFRQFHSWVQHHGHGWRVWDUW
XVLQJWKH/6)3DUDOOHOV\VWHP7KH\DUHFRPSLOLQJOLQNLQJDQGVXEPLWWLQJSDUDOOHO
DSSOLFDWLRQV7KHH[DPSOHXVHGLQWKLVFKDSWHULVDGLVWULEXWHGYHUVLRQRIWKH+HOOR
:RUOGSURJUDPQDPHGP\MREZULWWHQLQ&
7KLVFKDSWHUGLVFXVVHVWKHIROORZLQJWRSLFV
:ULWLQJD'LVWULEXWHG$SSOLFDWLRQ
&RPSLOLQJDQG/LQNLQJWKH$SSOLFDWLRQ
5XQQLQJWKH$SSOLFDWLRQ
,IWKHFRPPDQGVFDQQRWEHH[HFXWHGRUWKHPDQSDJHVFDQQRWEHYLHZHG
WKHDSSURSULDWHGLUHFWRULHVPD\QHHGWRDGGHGWRWKHV\VWHPVSDWKFKHFNZLWKWKH
V\VWHPDGPLQLVWUDWRU
Note
LSF Parallel User’s Guide
9
2
Getting Started
Writing a Distributed Application
Writing a Distributed Application
7KLVH[DPSOHSURJUDPZULWWHQLQ&LVDGLVWULEXWHGYHUVLRQRIWKH+HOOR:RUOG
SURJUDPQDPHGP\MRE8VHDQHGLWRUWRHQWHUWKHFRGHIRUWKLVDSSOLFDWLRQ$IWHU
WKHFRGHLVHQWHUHGVDYHLWLQDILOHQDPHGmyjob.c
/*
* File: myjob.c
*/
#include <stdio.h>
#include "mpi.h"
/* MPI header file */
int
main(int argc, char **argv)
{
int myrank;
int n_processes;
int srcrank;
int destrank;
char mbuf[512];
MPI_Status mstat;
/*
/*
/*
/*
/*
/*
Rank of this process */
Number of Processes */
Rank of the Sender */
Rank of the receiver */
Message buffer */
Return Status of an MPI operation */
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
MPI_Comm_size(MPI_COMM_WORLD, &n_processes);
if (myrank != 0) {
sprintf(mbuf, "Hello, from process %d!", myrank);
destrank = 0;
MPI_Send(mbuf, strlen(mbuf)+1, MPI_CHAR,
destrank, 90,MPI_COMM_WORLD);
} else {
for (srcrank = 1; srcrank < n_processes; srcrank++) {
MPI_Recv(mbuf, 512, MPI_CHAR,
srcrank, 90, MPI_COMM_WORLD,&mstat);
printf("From process %d: %s\n", srcrank, mbuf);
}
}
MPI_Finalize();
}
10 LSF Parallel User’s Guide
Getting Started
Compiling and Linking the Application
Compiling and Linking the Application
$IWHUWKHH[DPSOHSURJUDPLVHQWHUHGDQGVDYHGDVmyjob.cXVHWKHmpiccVFULSW
WRFRPSLOHDQGOLQNWKHDSSOLFDWLRQ7KHmpiccVFULSWLVXVHGLQDVLPLODUPDQQHU
WRRWKHU81,;EDVHG&FRPSLOHUV7KLVVFULSWSURYLGHVWKHRSWLRQVDQGVSHFLDO
OLEUDULHVQHHGHGWRFRPSLOHDQGOLQNDSDUDOOHODSSOLFDWLRQIRUWKH/6)3DUDOOHO
HQYLURQPHQW
Compile and Link
7RFRPSLOHDQGOLQNWKHVRXUFHFRGHLQWKHmyjob.cILOHLQRQHVWHSHQWHUWKH
IROORZLQJFRPPDQG
% mpicc myjob.c -o myjob
7KHELQDU\FUHDWHGLVFDOOHGmyjob
LSF Parallel User’s Guide 11
2
Getting Started
Running the Application
Running the Application
Submit to the LSF Batch System
7RVXEPLWWKHSDUDOOHODSSOLFDWLRQmyjobWRWKH/6)%DWFKV\VWHPUHTXHVWLQJWKUHH
SURFHVVRUVHQWHUWKHIROORZLQJFRPPDQG
% bsub -n 3 pam myjob
Job <1288> is submitted to default queue <normal>.
7KLVFRPPDQGFUHDWHVWKUHHSURFHVVHVDQGHDFKUXQVDQLQVWDQFHRI myjob7KH
bsubFRPPDQGKDVDQXPEHURIFRPPDQGOLQHRSWLRQVZKLFKDUHGLVFXVVHGLQ
PRUHGHWDLOLQ6XEPLWWLQJ%DWFK-REVRQSDJH7RYLHZWKHVWDWXVRIWKHSDUDOOHO
EDWFKMREHQWHUWKHIROORZLQJFRPPDQG
% bjobs
JOBID USER
1288 user1
STAT
PEND
QUEUE
normal
FROM_HOST
hopper
EXEC_HOST
host1
host2
host3
JOB_NAME
myjob
SUBMIT_TIME
Apr 16 14:43
7KHbjobsFRPPDQGKDVDQXPEHURIFRPPDQGOLQHRSWLRQVZKLFKDUHGLVFXVVHG
LQPRUHGHWDLOLQ0RQLWRULQJ-RE6WDWXVRQSDJH
Execute Interactively
7RLQWHUDFWLYHO\H[HFXWHWKHSDUDOOHODSSOLFDWLRQmyjobRQWKUHHSURFHVVRUVHQWHU
WKHIROORZLQJFRPPDQG
% pam -n 3 myjob
Server PAM contact address is klee:2801
From process 1: Hello, world from process 1!
From process 2: Hello, world from process 2!
From process 3: Hello, world from process 3!
TID
===
1
2
3
HOST_NAME
=========
host1
host2
host3
COMMAND_LINE
============
myjob
myjob
myjob
STATUS
========================
Completed
Completed
Completed
TERMINATION_TIME
==================
04/16/98 15:05:56
04/16/98 15:05:56
04/16/98 15:05:56
7KHpamFRPPDQGKDVDQXPEHURIFRPPDQGOLQHRSWLRQVZKLFKDUHGLVFXVVHG
LQPRUHGHWDLOLQ7KHSDP&RPPDQGRQSDJH
12 LSF Parallel User’s Guide
3 Building Parallel Applications
7KH/6)3DUDOOHOV\VWHPVSURYLGHVWRROVWRKHOSEXLOGDSDUDOOHODSSOLFDWLRQWRWDNH
IXOODGYDQWDJHRIWKH/6)%DWFKV\VWHP0RVWSDUDOOHODSSOLFDWLRQVFDQEHUHXVHG
E\VLPSO\UHOLQNLQJZLWKWKH3$0DZDUH03,OLEUDU\LQVRPHLQVWDQFHVWKHUHPD\
QRWHYHQEHDQHHGWRUHFRPSLOH
7KLVFKDSWHUGLVFXVVHVWKHEDVLFVWHSVLQEXLOGLQJDSDUDOOHODSSOLFDWLRQ:HGLVFXVV
WKHEDVLFVWUXFWXUHRIWKHDSSOLFDWLRQDQGKRZLWLVFRPSLOHGDQGOLQNHG
7KLVFKDSWHUIRFXVHVRQEXLOGLQJDSDUDOOHODSSOLFDWLRQWRPDNHRSWLPDOXVH
RIWKH/6)%DWFKV\VWHP,WDVVXPHVIDPLOLDULW\ZLWKWKH/6)6XLWHRISURGXFWVDQG
VWDQGDUG03,7KHUHIRUHLWGRHVQRWGLVFXVVZULWLQJ03,SURJUDPV
Note
7KLVFKDSWHUFRQWDLQVWKHIROORZLQJWRSLFV
,QFOXGLQJWKH+HDGHU)LOH
&RPSLOLQJDQG/LQNLQJ %XLOGLQJD+HWHURJHQHRXV3DUDOOHO$SSOLFDWLRQ
LSF Parallel User’s Guide 13
3
Building Parallel Applications
Including the Header File
Including the Header File
$VHWRI3$0DZDUHKHDGHUILOHVDUHLQFOXGHGZLWKWKH/6)3DUDOOHOV\VWHP
LQVWDOODWLRQ7KH\DUHW\SLFDOO\ORFDWHGLQWKHLSF_INCLUDEDIR/lsf/mpi/
GLUHFWRU\7KHKHDGHUILOHVFRQWDLQWKH03,GHILQLWLRQVPDFURVDQGIXQFWLRQ
SURWRW\SHVQHFHVVDU\IRUXVLQJWKH/6)3DUDOOHOV\VWHP
Include Syntax
7KHLQFOXGHV\QWD[PXVWEHSODFHGDWWKHWRSRIDQ\SDUDOOHODSSOLFDWLRQWKDWFDOOV
03,URXWLQHV7KHLQFOXGHVWDWHPHQWORRNVOLNHWKLV
,Q&DSSOLFDWLRQV
#include <mpi.h>
,Q)RUWUDQDSSOLFDWLRQV
INCLUDE ‘mpif.h’
,IWKHKHDGHUILOHVDUHQRWORFDWHGLQWKHLSF_INCLUDEDIR/lsf/mpi/
GLUHFWRU\FKHFNZLWK\RXUV\VWHPDGPLQLVWUDWRU
Note
14 LSF Parallel User’s Guide
Building Parallel Applications
Compiling and Linking
Compiling and Linking
7KH/6)3DUDOOHOV\VWHPSURYLGHVDVHWRIVFULSWVWKDWKHOSZLWKWKHFUHDWLRQRI
H[HFXWDEOHREMHFWV7KH\DUHmpiccIRU&SURJUDPVDQGmpif77IRU)RUWUDQ
SURJUDPV7KHVHVFULSWVSURYLGHWKHRSWLRQVDQGVSHFLDOOLEUDULHVQHHGHGWRFRPSLOH
DQGOLQN03,SURJUDPVIRUXVHZLWKWKH/6)3DUDOOHOV\VWHP$SSOLFDWLRQVDUHOLQNHG
WRV\VWHPGHSHQGHQWOLEUDULHVDQGWKHDSSURSULDWH03,OLEUDU\
C Programs
7KH/6)3DUDOOHO&FRPSLOHUmpiccLVXVHGWRFRPSLOH03,&VRXUFHILOHV,WLV
XVHGLQDVLPLODUPDQQHUWRRWKHU81,;EDVHG&FRPSLOHUV
)RUH[DPSOH7RFRPSLOHWKHVDPSOHSURJUDPFRQWDLQHGLQDILOHmyjob.cHQWHU
% mpicc -c myjob.c
7KLVFRPPDQGSURGXFHVWKHmyjob.oWKDWFRQWDLQVWKHREMHFWFRGHIRUWKLV/6)
3DUDOOHOVRXUFHILOH
7ROLQNWKHmyjob.oREMHFWILOHZLWKWKH/6)3DUDOOHOOLEUDULHVWRFUHDWHDQ
H[HFXWDEOHHQWHU
% mpicc -o myjob myjob.o
$VZLWKPRVW&FRPSLOHUVWKH-oIODJVSHFLILHVWKDWWKHQDPHRIWKHH[HFXWDEOH
SURGXFHGE\WKHOLQNHULVWREHmyjob
7KH&VRXUFHILOHFDQEHFRPSLOHGDQGOLQNHGLQRQHVWHSXVLQJWKHIROORZLQJ
FRPPDQG
% mpicc myjob -o myjob
Fortran 77 Programs
7KH/6)3DUDOOHO)RUWUDQFRPSLOHUmpif77LVXVHGWRFRPSLOH03,)RUWUDQ
VRXUFHILOHV,WLVXVHGLQDVLPLODUPDQQHUWRRWKHU81,;EDVHG)RUWUDQ
FRPSLOHUV
)RUH[DPSOH7RFRPSLOHWKHVDPSOHSURJUDPFRQWDLQHGLQDILOHmyjob.fHQWHU
% mpif77 -c myjob.f
LSF Parallel User’s Guide 15
3
Building Parallel Applications
Compiling and Linking
7KLVFRPPDQGSURGXFHVWKHmyjob.oWKDWFRQWDLQVWKHREMHFWFRGHIRUWKLV/6)
3DUDOOHOVRXUFHILOH
7ROLQNWKHmyjob.oREMHFWILOHZLWKWKH/6)3DUDOOHOOLEUDULHVWRFUHDWHDQ
H[HFXWDEOHHQWHU
% mpif77 -o myjob myjob.o
$VZLWKPRVW)RUWUDQFRPSLOHUVWKH-oIODJVSHFLILHVWKDWWKHQDPHRIWKH
H[HFXWDEOHSURGXFHGE\WKHOLQNHULVWREHmyjob
7KH)RUWUDQVRXUFHILOHFDQEHFRPSLOHGDQGOLQNHGLQRQHVWHSXVLQJWKH
IROORZLQJFRPPDQG
% mpif77 myjob -o myjob
16 LSF Parallel User’s Guide
Building Parallel Applications
Building a Heterogeneous Parallel Application
Building a Heterogeneous Parallel Application
7KH/6)3DUDOOHOV\VWHPSURYLGHVDKRVWW\SHVXEVWLWXWLRQIDFLOLW\WRDOORZD
KHWHURJHQHRXVPXOWLSOHDUFKLWHFWXUHGLVWULEXWHGDSSOLFDWLRQWREHVXEPLWWHGWRWKH
/6)%DWFKV\VWHP7KHIROORZLQJVWHSVRXWOLQHKRZWREXLOGDQGGHSOR\D
KHWHURJHQHRXVDSSOLFDWLRQ
'HVLJQWKHSDUDOOHODSSOLFDWLRQ
&RPSLOHWKHDSSOLFDWLRQRQDOO/6)KRVWW\SHDUFKLWHFWXUHVWKDWZLOOEHXVHGWR
VXSSRUWWKLVDSSOLFDWLRQ
7KHELQDULHVPXVWHLWKHUEHQDPHGZLWKYDOLG/6)KRVWW\SHH[WHQVLRQV
RUSODFHGLQGLUHFWRULHVQDPHGZLWKYDOLG/6)KRVWW\SHSDWKQDPHV
Note
3ODFHELQDULHVLQWKHDSSURSULDWHVKDUHGILOHV\VWHPRUGLVWULEXWHWKHP
DFFRUGLQJO\
8VHWKH%aDQQRWDWLRQWRVXEPLWWKHSDUDOOHODSSOLFDWLRQWRWKH/6)%DWFK
V\VWHP
LSF Host Type Naming Convention
%LQDULHVPXVWEHFRPSLOHGRQWKHWDUJHWKRVWW\SHDUFKLWHFWXUHV7KHELQDU\PXVW
EHQDPHGXVLQJDYDOLG/6)KRVWW\SHVWULQJDVWKHH[WHQVLRQWRLWVQDPHRUWKH
QDPHRIDGLUHFWRU\LQLWVSDWKlshostsGLVSOD\VDOLVWRIYDOLG/6)KRVWW\SHV
:KHQWKH%aQRWDWLRQLVXVHGWRVXEPLWDSDUDOOHODSSOLFDWLRQWRWKH/6)%DWFK
V\VWHPWKHWDUJHWKRVWW\SHVWULQJLVVXEVWLWXWHG
$OOELQDULHVIRUDVSHFLILFDSSOLFDWLRQPXVWEHQDPHGXVLQJWKHVDPHKRVWW\SH
VXEVWLWXWLRQIRUPDWLHELQDU\H[WHQVLRQRUSDWKQDPH
([DPSOH7KHIROORZLQJELQDULHVDUHQDPHGZLWKDSSURSULDWHKRVWW\SHH[WHQVLRQV
WRLGHQWLI\WKHWDUJHWSODWIRUPRQZKLFKWKH\DUHWRUXQ7KHVHELQDULHVDUHQDPHG
WRXVH6XQ6RODULVDQG56DUFKLWHFWXUHPDFKLQHV
•
•
myjob.SUNSOL
myjob.RS6K
LSF Parallel User’s Guide 17
3
Building Parallel Applications
Building a Heterogeneous Parallel Application
([DPSOH7KHIROORZLQJELQDULHVDUHQDPHGZLWKDSSURSULDWHSDWKQDPHVWRLGHQWLI\
WKHWDUJHWSODWIRUPRQZKLFKWKH\DUHWRUXQ7KHVHELQDULHVDUHQDPHGWRXVH6XQ
6RODULVDQG56DUFKLWHFWXUHPDFKLQHV
•
•
/user/batch/SUNSOL/myjob
/user/batch/RS6K/myjob
%a Notation
$IWHUDSDUDOOHODSSOLFDWLRQLVVXEPLWWHGWRWKH/6)%DWFKV\VWHPWKH3DUDOOHO
$SSOLFDWLRQ0DQDJHU3$0UHSODFHVWKH%aDQQRWDWLRQZLWKWKHDSSURSULDWH/6)
KRVWW\SHVWULQJ3$0WKHQODXQFKHVWKHLQGLYLGXDOWDVNVRIWKHDSSOLFDWLRQRQWKH
UHPRWHKRVWVXVLQJWKHFRUUHFWELQDULHV
Note 8VHWKHlshostsFRPPDQGWRGHWHUPLQHZKLFK/6)KRVWVDUHDYDLODEOH)RU
H[DPSOH
% lshosts
HOST_NAME type
host1
SUNSOL
host2
RS6K
model
SunSparc
IBM350
cpuf
6.0
7.0
ncpus maxmem maxswp server
RESOURCES
1
64M
112M
Yes (solaris cserver)
1
64M
124M
Yes
(cserver aix)
([DPSOH7RVXEPLWWKHmyjobDSSOLFDWLRQIURPWKHVDPHGLUHFWRU\XVLQJ/6)KRVW
W\SHH[WHQVLRQVWKHIROORZLQJFRPPDQGLVXVHG
% pam -n 2 myjob.%a
3$0ZLOOPDNHWKHIROORZLQJVXEVWLWXWLRQVIRUWKH%aQRWDWLRQ
•
•
myjob.SUNSOL
myjob.RS6K
([DPSOH7RVXEPLWWKHmyjobDSSOLFDWLRQIURPGLIIHUHQWGLUHFWRULHVXVLQJKRVW
W\SHSDWKQDPHVWKHIROORZLQJFRPPDQGLVEHXVHG
% pam -n 2 /user/batch/%a/myjob
3$0ZLOOPDNHWKHIROORZLQJVXEVWLWXWLRQVIRUWKH%aQRWDWLRQ
•
•
/user/batch/SUNSOL/myjob
/user/batch/RS6K/myjob
18 LSF Parallel User’s Guide
4 Submitting Parallel Applications
7KLVFKDSWHUGHVFULEHVKRZWRVXEPLWDQGLQWHUDFWZLWKSDUDOOHODSSOLFDWLRQVLQWKH
/6)%DWFKV\VWHP
$QH[WHQVLYHDQGIOH[LEOHVHWRIWRROVLVSURYLGHGWKDWDOORZVSDUDOOHODSSOLFDWLRQV
WREHVXEPLWWHGWKURXJKWKH/6)%DWFKV\VWHP3DUDOOHODSSOLFDWLRQVFDQDOVREH
H[HFXWHGLQWHUDFWLYHO\XQGHUFRQWURORIWKH3DUDOOHO$SSOLFDWLRQ0DQJHU3$0
7KHVHWRROVDOORZWKHVSHFLILFDWLRQRIKRZZKHQDQGZKHUHDSDUDOOHODSSOLFDWLRQ
LVWREHUXQ
7KLVFKDSWHUGLVFXVVHVWKHIROORZLQJWRSLFV
-RE6XEPLVVLRQ0HWKRGV %DWFK([HFXWLRQ
%DWFK-RE6WDWXV 6XEPLWWLQJ%DWFK-REV 6XVSHQGLQJ-REV 5HVXPLQJ-REV
0RQLWRULQJ-RE6WDWXV
7HUPLQDWLQJ-REV
5XQQLQJ+HWHURJHQHRXV3DUDOOHO$SSOLFDWLRQV ,QWHUDFWLYH([HFXWLRQ
7KHSDP&RPPDQG 3URFHVV6WDWXV5HSRUW *HWWLQJ+RVW,QIRUPDWLRQ LSF Parallel User’s Guide 19
4
Submitting Parallel Applications
Job Submission Methods
Job Submission Methods
7KH/6)3DUDOOHOV\VWHPVXSSRUWVEDWFKVXEPLVVLRQRISDUDOOHODSSOLFDWLRQVEDWFK
MREVXVLQJWKHIDFLOLWLHVRIWKH/6)%DWFK6\VWHP,QWHUDFWLYHH[HFXWLRQRISDUDOOHO
DSSOLFDWLRQVLVDOVRVXSSRUWHGXQGHUFRQWURORIWKH3DUDOOHO$SSOLFDWLRQ0DQDJHU
3$0
Batch Execution
:KHQVXEPLWWLQJDSDUDOOHOEDWFKMREWKH/6)3DUDOOHOV\VWHPXVHVWKHDGYDQFHG
IHDWXUHVRIWKH/6)%DWFKV\VWHPWRVHOHFWVXEPLWDQGLQWHUDFWZLWKWKHLQGLYLGXDO
WDVNVRIWKHSDUDOOHOEDWFKMRE7KHEDWFKMRELVVXEPLWWHGWRDTXHXHXVLQJWKHbsub
FRPPDQGDQGWKH/6)%DWFKV\VWHPDWWHQGVWRWKHGHWDLOV
$SDUDOOHOEDWFKMRELVVXEPLWWHGWRDTXHXHZKHUHLWZDLWVXQWLOLWUHDFKHVWKHIURQW
RIWKHTXHXHDQGWKHDSSURSULDWHUHVRXUFHVEHFRPHDYDLODEOH7KHQWKHEDWFKMRE
ZLOOEHGLVSDWFKHGWRWKHPRVWVXLWDEOHKRVWVIRUH[HFXWLRQ7KLVVRSKLVWLFDWHG
TXHXLQJV\VWHPDOORZVEDWFKMREVWRUXQDVVRRQDVWKHVXLWDEOHKRVWUHVRXUFHV
EHFRPHVDYDLODEOH
7KHEDWFKMREPD\QRWEHUXQLPPHGLDWHO\LWPD\TXHXHGXQWLOWKH
DSSURSULDWHUHVRXUFHVEHFRPHDYDLODEOH
Note
7RXVHWKHbsubFRPPDQGWRVXEPLWDSDUDOOHOEDWFKMREWRWKH/6)%DWFKV\VWHP
VHH6XEPLWWLQJ%DWFK-REVRQSDJH
Interactive Execution
:KHQLQWHUDFWLYHO\H[HFXWLQJDSDUDOOHOEDWFKMREWKHpamFRPPDQGLVXVHGWR
LQYRNH3$0:KHQVXEPLWWLQJEDWFKMREVXVLQJWKHpamFRPPDQGWKH/6)%DWFK
V\VWHPLVE\SDVVHGWKHMREVDUHQRWTXHXHG%DWFKMREVDUHUXQLPPHGLDWHO\XSRQ
HQWHULQJWKHFRPPDQGLIWKHVSHFLILHGUHVRXUFHUHTXLUHPHQWVDUHPHW,IWKH
UHVRXUFHVDUHQRWDYDLODEOHWKHMRELVQRWUXQ
6LQFHWKHMREVGRQRWZDLWLQWHUDFWLYHMREH[HFXWLRQLVEHQHILFLDOIRUGHEXJJLQJ
SDUDOOHODSSOLFDWLRQV'LUHFWLQWHUDFWLRQLVVXSSRUWHG$OOWKHLQSXWDQGRXWSXWLV
KDQGOHGWUDQVSDUHQWO\EHWZHHQWKHORFDODQGH[HFXWLRQKRVWV
7RXVHWKHpamFRPPDQGWRH[HFXWHDSDUDOOHOEDWFKMRELQWHUDFWLYHO\VHH,QWHUDFWLYH
([HFXWLRQRQSDJH
20 LSF Parallel User’s Guide
Submitting Parallel Applications
Job Submission Methods
LSF Parallel and LSF Batch Commands
7KH/6)%DWFKDQG/6)3DUDOOHOSURGXFWVSURYLGHFRPPDQGVDQGPDQSDJHVIRU
WKHVHFRPPDQGV
,IWKHVHFRPPDQGVFDQQRWEHH[HFXWHGRUWKHPDQSDJHVFDQQRWEHYLHZHGWKH
DSSURSULDWHGLUHFWRULHVPD\QHHGWREHDGGHGWRWKHV\VWHPVSDWKFKHFNZLWKWKH
V\VWHPDGPLQLVWUDWRU
LSF Parallel User’s Guide 21
4
Submitting Parallel Applications
Batch Execution
Batch Execution
7KH/6)3DUDOOHOV\VWHPXVHVWKHIHDWXUHVRIWKH/6)%DWFKV\VWHPWRVHOHFWWKH
PRVWVXLWDEOHKRVWVVXEPLWDQGLQWHUDFWZLWKSDUDOOHOEDWFKMREV7KHEDWFKMRELV
VXEPLWWHGWRDTXHXHXVLQJWKHbsubFRPPDQGDVGHVFULEHGLQ6XEPLWWLQJ%DWFK-REV
RQSDJHDQGWKH/6)%DWFKDQG/6)3DUDOOHOV\VWHPVDWWHQGWRWKHUHVW
/LNHVHULDOEDWFKMREVSDUDOOHOEDWFKMREVSDVVWKURXJKPDQ\VWDWHV6HH%DWFK-RE
6WDWXVRQSDJH
7KLVSDUWRIWKHFKDSWHUGLVFXVVHVWKHIROORZLQJWRSLFV
%DWFK-RE6WDWXV 6XEPLWWLQJ%DWFK-REV 6XVSHQGLQJ-REV 5HVXPLQJ-REV
0RQLWRULQJ-RE6WDWXV
7HUPLQDWLQJ-REV
5XQQLQJ+HWHURJHQHRXV3DUDOOHO$SSOLFDWLRQV 22 LSF Parallel User’s Guide
Submitting Parallel Applications
Batch Job Status
Batch Job Status
(DFKEDWFKMREVXEPLWWHGWRWKH/6)%DWFKV\VWHPSDVVHVWKURXJKDVHULHVRIVWDWHV
XQWLOWKHMREFRPSOHWHVQRUPDOO\VXFFHVVRUDEQRUPDOO\IDLOXUH7KHbjobs
FRPPDQGDOORZVWKHVWDWXVRIWKHEDWFKMREVWREHPRQLWRUHGVHH0RQLWRULQJ-RE
6WDWXVRQSDJH7KHDELOLW\WRPRQLWRUEDWFKMREVWDWXVH[WHQGVWRWKHLQGLYLGXDO
SURFHVVHVWDVNVRIWKHSDUDOOHODSSOLFDWLRQ
)LJXUH%DWFK-RE6WDWH'LDJUDP
suitable host found
normal
completion
bsub
PEND
RUN
migration
DONE
bresume
host OK
host overloaded
bstop
PSUSP
SSUSP
bstop
bkill
EXIT
bkill
or abnormal
exit
bresume
USUSP
bkill
LSF Parallel User’s Guide 23
4
Submitting Parallel Applications
Batch Job Status
)LJXUHVKRZVWKHSRVVLEOHVWDWHVDEDWFKMREFDQSDVVWKURXJKZKHQVXEPLWWHGWR
WKH/6)%DWFKV\VWHP7KHGLDJUDPDOVRVKRZVWKHDFWLYLWLHVDQGFRPPDQGVWKDW
FDXVHWKHVWDWHWUDQVLWLRQV7KHEDWFKMREVWDWHVDUHGHVFULEHGLQ7DEOH
7DEOH%DWFK-RE6WDWH'HVFULSWLRQV
6WDWH
'HVFULSWLRQ
3(1' $EDWFKMRELVSHQGLQJZKHQLWLVVXEPLWWHGXVLQJWKHbsubFRPPDQG
DQGZDLWLQJLQDTXHXH,WUHPDLQVSHQGLQJXQWLOLWPRYHVWRWKHKHDGRI
WKHTXHXHDQGDOOFRQGLWLRQVIRULWVH[HFXWLRQDUHPHW7KHFRQGLWLRQV
PD\LQFOXGH
‡
‡
‡
6WDUWWLPHVSHFLILHGE\WKHXVHUZKHQWKHMRELVVXEPLWWHG
/RDGFRQGLWLRQVRQTXDOLILHGKRVWV
7LPHZLQGRZVGXULQJZKLFK
‡ 7KHTXHXHFDQGLVSDWFKMREV
‡ 4XDOLILHGKRVWVFDQDFFHSWMREV
‡ 5HODWLYHSULRULW\WRRWKHUXVHUVDQGMREV
‡ $YDLODELOLW\RIWKHVSHFLILHGUHVRXUFHV
581
$EDWFKMRELVUXQQLQJZKHQLWKDVEHHQGLVSDWFKHGWRDKRVW
'21( $EDWFKMRELVGRQHZKHQLWKDVQRUPDOO\FRPSOHWHGLWVH[HFXWLRQ
36863 7KHMRERZQHURUWKH/6)$GPLQLVWUDWRUFDQVXVSHQGXVLQJWKHbstop
FRPPDQGDEDWFKMREZKLOHLWLVSHQGLQJ
$OVRWKHMRERZQHURUWKH/6)$GPLQLVWUDWRUFDQUHVXPHXVLQJWKH
bresumeFRPPDQGDEDWFKWKDWLVLQWKH36863VWDWHWKHQWKHEDWFK
MREVWDWHWUDQVLWLRQVWR3(1'
86863 7KHMRERZQHURUWKH/6)$GPLQLVWUDWRUFDQVXVSHQGXVLQJWKHbstop
FRPPDQGDEDWFKMREDIWHULWKDVEHHQGLVSDWFKHG
$OVRWKHRZQHURUWKH/6)$GPLQLVWUDWRUFDQUHVXPHXVLQJWKH
bresumeFRPPDQGDEDWFKWKDWLVLQWKH86863VWDWHWKHQWKHEDWFK
MREVWDWHWUDQVLWLRQVWR66863
24 LSF Parallel User’s Guide
Submitting Parallel Applications
Batch Job Status
7DEOH%DWFK-RE6WDWH'HVFULSWLRQV
6WDWH
66863
'HVFULSWLRQ
$EDWFKMREFDQEHVXVSHQGHGE\WKH/6)%DWFKV\VWHPDIWHULWKDVEHHQ
GLVSDWFKHG7KLVLVGRQHLIWKHORDGRQWKHH[HFXWLRQKRVWRUKRVWV
EHFRPHVWRRKLJKLQRUGHUWRPD[LPL]HKRVWSHUIRUPDQFHRUWR
JXDUDQWHHLQWHUDFWLYHUHVSRQVHWLPH
7KH/6)%DWFKV\VWHPVXVSHQGVEDWFKMREVDFFRUGLQJWRWKHLUSULRULW\
XQOHVVWKHVFKHGXOLQJSROLF\DVVRFLDWHGZLWKWKHMREGLFWDWHVRWKHUZLVH
$EDWFKMREPD\DOVREHVXVSHQGHGLIWKHMRETXHXHKDVDWLPHZLQGRZ
DQGWKHFXUUHQWWLPHH[FHHGVWKHZLQGRZ
(;,7
7KH/6)%DWFKV\VWHPFDQODWHUUHVXPHDV\VWHPVXVSHQGHG66863
MRELIWKHORDGFRQGLWLRQRQWKHH[HFXWLRQKRVWGHFUHDVHVRUWKHWLPH
ZLQGRZRIWKHTXHXHRSHQV
$EDWFKMREFDQWHUPLQDWHDEQRUPDOO\IDLOIURPDQ\VWDWHIRUPDQ\
UHDVRQV$EQRUPDOMREWHUPLQDWLRQFDQRFFXUZKHQ
‡
‡
‡
‡
&DQFHOOHGXVLQJWKHbkillFRPPDQGE\RZQHURU/6)
DGPLQLVWUDWRUZKLOHLQ3(1'581RU86863VWDWH
$ERUWHGE\/6)EHFDXVHMREFDQQRWEHGLVSDWFKHGEHIRUHD
WHUPLQDWLRQGHDGOLQH
)DLOVWRVWDUWVXFFHVVIXOO\HJWKHZURQJH[HFXWDEOHZDVVSHFLILHG
DWWLPHRIMREVXEPLVVLRQ
&UDVKHVGXULQJH[HFXWLRQ
Parallel Batch Job Behavior
‡
‡
:KHQRQHWDVNH[LWVZLWKDQRQH]HURUHWXUQYDOXHDOOWKHRWKHUWDVNVZLOOUXQ
XQWLOWKH\FRPSOHWH'21(RUIDLO(;,7
:KHQRQHWDVNLVNLOOHGE\DVLJQDORUFRUHGXPSVDOOWKHRWKHUWDVNVZLOOEH
VKXWGRZQ
LSF Parallel User’s Guide 25
4
Submitting Parallel Applications
Submitting Batch Jobs
Submitting Batch Jobs
The bsub Command
7KHbsubFRPPDQGLVXVHGWRVXEPLWSDUDOOHOEDWFKMREVWRWKH/6)%DWFKV\VWHP
7KHV\QWD[IRUXVLQJbsubZKHQVXEPLWWLQJSDUDOOHODSSOLFDWLRQVLVWKHVDPHDVWKH
/6)%DWFKV\VWHPZLWKWKHDGGLWLRQRIWKHpamRSWLRQ
% bsub [options] pam [options] job
The pam Option
7KHpamRSWLRQVXVHGZLWKWKHbsubFRPPDQGDUHDVXEVHWRIWKHpamFRPPDQG
RSWLRQVVHH7KHSDP&RPPDQGRQSDJH6LQFHWKH/6)%DWFKV\VWHPGRHVDOORI
WKHUHVRXUFHDOORFDWLRQDQGVFKHGXOLQJWKHpamRSWLRQVPIDQGQDUHQRW
QHFHVVDU\DQGDUHLJQRUHGE\WKHbsubFRPPDQG7KHV\QWD[IRUbsub pamLV
pam [-h][-V][-t][-v]
7KH bsub pam RSWLRQVDUH
2SWLRQ
K
9
W
Y
'HVFULSWLRQ
3ULQWFRPPDQGXVDJHWRVWDQGDUGHUURUDQGH[LW
3ULQW/6)YHUVLRQWRVWDQGDUGHUURUDQGH[LW
6XSSUHVVWKHSULQWLQJRIWKHSURFHVVVWDWXVVXPPDU\RQMRE
FRPSOHWLRQ
6SHFLILHVWKHMRELVWREHUXQLQYHUERVHPRGH7KHQDPHVRI
WKHVHOHFWHGKRVWVDUHGLVSOD\HG
([DPSOH7KHIROORZLQJFRPPDQGVXEPLWVDSDUDOOHOEDWFKMREQDPHGmyjobWR
WKH/6)%DWFKV\VWHPDQGUHTXHVWVIRXUSURFHVVRUVRIDQ\W\SHWRUXQWKHMRE
% bsub -n 4 pam myjob
:KHQWKHSDUDOOHOEDWFKMREQDPHGmyjobLVVXEPLWWHGWRWKH/6)%DWFKV\VWHP
DQGGLVSDWFKHGWRKRVWKRVWKRVWDQGKRVWWKHbjobsFRPPDQGZLOOGLVSOD\
26 LSF Parallel User’s Guide
Submitting Parallel Applications
Submitting Batch Jobs
% bjobs
JOBID USER
713
user1
STAT
RUN
QUEUE
batch
FROM_HOST
host99
EXEC_HOST
host1
host2
host3
host4
JOB_NAME
myjob
SUBMIT_TIME
Sep 12 16:30
LSF Parallel User’s Guide 27
4
Submitting Parallel Applications
Suspending Jobs
Suspending Jobs
The bstop Command
7KHbstopFRPPDQGLVXVHGWRVXVSHQGSDUDOOHOEDWFKMREVUXQQLQJLQWKH/6)
%DWFKV\VWHP7KHV\QWD[IRUXVLQJWKHbstopFRPPDQGLQWKH/6)3DUDOOHOV\VWHP
LV
% bstop jobId
([DPSOH7KHIROORZLQJFRPPDQGVXVSHQGVWKHSDUDOOHOEDWFKMREQDPHGmyjob
UXQQLQJLQWKH/6)%DWFKV\VWHPZLWKMRELGRI
% bstop 713
:KHQWKHSDUDOOHOEDWFKMREQDPHGmyjobLVVXVSHQGHGWKHbjobsFRPPDQGZLOO
GLVSOD\WKHEDWFKMREVWDWHRI86863
% bjobs
JOBID USER
713
user1
STAT QUEUE
USUSP batch
28 LSF Parallel User’s Guide
FROM_HOST
host99
EXEC_HOST
host1
host2
host3
host4
JOB_NAME
myjob
SUBMIT_TIME
Sep 12 16:32
Submitting Parallel Applications
Resuming Jobs
Resuming Jobs
The bresume Command
7KHbresumeFRPPDQGLVXVHGWRUHVXPHVXVSHQGHGSDUDOOHOEDWFKMREVUXQQLQJ
LQWKH/6)%DWFKV\VWHP7KHV\QWD[IRUXVLQJWKHbresumeFRPPDQGLQWKH/6)
3DUDOOHOV\VWHPLV
% bresume jobID
([DPSOH7KHIROORZLQJFRPPDQGUHVXPHVWKHVXVSHQGHGSDUDOOHOEDWFKMREQDPHG
myjobUXQQLQJLQWKH/6)%DWFKV\VWHPZLWKMRE,'RI
% bresume 713
:KHQWKHSDUDOOHOEDWFKMREQDPHGmyjobLVUHVXPHGWKHbjobsFRPPDQGZLOO
GLVSOD\WKHEDWFKMREVWDWHRI581RU3(1'
% bjobs
JOBID USER
713
user1
STAT
RUN
QUEUE
batch
FROM_HOST
host99
EXEC_HOST
host1
host2
host3
host4
JOB_NAME
myjob
SUBMIT_TIME
Sep 12 16:34
LSF Parallel User’s Guide 29
4
Submitting Parallel Applications
Monitoring Job Status
Monitoring Job Status
The bjobs Command
7KHbjobsFRPPDQGLVXVHGWRYLHZWKHUXQQLQJVWDWXVDQGUHVRXUFHXVDJHRI
SDUDOOHOEDWFKMREVUXQQLQJLQWKH/6)%DWFKV\VWHP7KHV\QWD[IRUXVLQJWKH bjobs
FRPPDQGLQWKH/6)3DUDOOHOV\VWHPLV
% bjobs [options]
([DPSOH7KHIROORZLQJFRPPDQGGLVSOD\VWKHUXQQLQJVWDWXVDQGUHVRXUFHXVDJH
RIWKHMREVUXQQLQJLQWKH/6)%DWFKV\VWHP
% bjobs
JOBID USER
713
user1
STAT
RUN
QUEUE
batch
FROM_HOST
host99
EXEC_HOST
host1
host2
host3
host4
JOB_NAME
myjob
SUBMIT_TIME
Sep 12 16:34
([DPSOH7KHIROORZLQJFRPPDQGXVHVWKH-lRSWLRQWRGLVSOD\UXQWLPHUHVRXUFH
XVDJH&38PHPRU\DQGVZDSDVZHOODVWKHUXQQLQJVWDWXVRIWKHMREVUXQQLQJLQ
WKH/6)%DWFKV\VWHP
% bjobs -l
Job Id <713>, User, Project, Status, Queue, Interactive pseudo-terminal mode,
Command <myjob>
Thu Sep 12 16:39:17: Submitted from host <host99>, CWD <$HOME/Work/utopia/pass/
pam>, 2-4 Processors Requested;
Thu Sep 12 16:39:18: Started on 4 Hosts/Processors host1 host2 host3 host4,
Execution Home < /pcc/s/user1, Execution CWD < /pcc/s/user1/W
ork/utopia/pass/pam;
Thu Sep 12 16:40:41: Resource usage collected.
The CPU time used is 2 seconds.
MEM: 281 Kbytes; SWAP: 367 Kbytes
PGIDs: 4 PIDs: 4
SCHEDULING PARAMETERS:
r15s
r1m r15m
ut
pg
io
ls
it
tmp
swp
mem
loadSched
loadStop
nresj
loadSched
loadStop
-
30 LSF Parallel User’s Guide
Submitting Parallel Applications
Terminating Jobs
Terminating Jobs
The bkill Command
7KHbkillFRPPDQGLVXVHGWRWHUPLQDWHSDUDOOHOEDWFKMREVUXQQLQJLQWKH/6)
%DWFKV\VWHP7KHV\QWD[IRUXVLQJWKHbkillFRPPDQGLQWKH/6)3DUDOOHOV\VWHP
LV
% bkill jobID [options]
([DPSOH7KHIROORZLQJFRPPDQGWHUPLQDWHVWKHSDUDOOHOEDWFKMREQDPHGmyjob
UXQQLQJLQWKH/6)%DWFKV\VWHPZLWKDMRE,'RI
% bkill 713
:KHQWKHSDUDOOHOEDWFKMREQDPHGmyjobLVWHUPLQDWHGWKHbjobsFRPPDQGZLOO
GLVSOD\WKHEDWFKMREVWDWHRI(;,7
% bkill 713
JOBID USER
713
user1
STAT
EXIT
QUEUE
batch
FROM_HOST
host99
EXEC_HOST
host1
host2
host3
host4
JOB_NAME
myjob
SUBMIT_TIME
Sep 12 16:30
7KHWLPHWDNHQWRWHUPLQDWHDSDUDOOHOEDWFKMREYDULHVDQGGHSHQGVRQWKH
QXPEHURISDUDOOHOSURFHVVHV
Note
LSF Parallel User’s Guide 31
4
Submitting Parallel Applications
Running Heterogeneous Parallel Applications
Running Heterogeneous Parallel Applications
7KH/6)3DUDOOHOV\VWHPSURYLGHVDQ/6)KRVWW\SHVXEVWLWXWLRQIDFLOLW\WRDOORZD
KHWHURJHQHRXVPXOWLSOHDUFKLWHFWXUHGLVWULEXWHGDSSOLFDWLRQWREHVXEPLWWHGWRWKH
/6)%DWFKV\VWHP
$VVXPSWLRQV
7KHELQDU\ZLOOUXQRQHDFKVSHFLILHGSODWIRUPRUDELQDU\H[LVWVIRUHDFK
SODWIRUP
7KHELQDULHVIRUWKH3DUDOOHODSSOLFDWLRQDUHVSHFLILHGXVLQJWKH%aQRWDWLRQ
IRUPDWVHH%XLOGLQJD+HWHURJHQHRXV3DUDOOHO$SSOLFDWLRQRQSDJH
([DPSOH8VLQJWKH/6)KRVWW\SHH[WHQVLRQIRUPDWWRVSHFLI\WKHEDWFKMRE
QDPHGmyjobWRUXQRQDQ\WZRDYDLODEOHSURFHVVRUVKDYLQJHLWKHU6XQ6RODULV
68162/RU5656.DUFKLWHFWXUHVWKHIROORZLQJFRPPDQGFDQEHXVHG
% bsub -n 2 pam myjob.%a
7RVSHFLI\68162/DQG56.LQDQHQYLURQPHQWZLWKRWKHUDUFKLWHFWXUHVWKH
IROORZLQJFRPPDQGLVVSHFLILHGZLWKWKH-RUHVRXUFHRSWLRQ
% bsub -n 2 -R "type==SUNSOL || type==RS6K" pam myjob.%a
)RUERWKWKHVHH[DPSOHVWKH3DUDOOHO$SSOLFDWLRQ0DQDJHU3$0VXEVWLWXWHVWKH
%aQRWDWLRQZLWKWKHFRUUHFW/6)KRVWW\SHH[WHQVLRQ7KHELQDULHVXVHGDUHQDPHG
•
•
myjob.SUNSOL
myjob.RS6K
32 LSF Parallel User’s Guide
Submitting Parallel Applications
Running Heterogeneous Parallel Applications
([DPSOH8VLQJWKH/6)KRVWW\SHSDWKQDPHIRUPDWWRVSHFLI\WKHEDWFKMRE
QDPHGmyjobWRUXQRQDQ\WZRSURFHVVRUVKDYLQJHLWKHU68162/RU56.
DUFKLWHFWXUHVWKHIROORZLQJFRPPDQGFDQEHXVHG
% bsub -n 2 pam /user/batch/%a/myjob
7RVSHFLI\68162/DQG56.LQDQHQYLURQPHQWZLWKRWKHUDUFKLWHFWXUHVWKH
IROORZLQJFRPPDQGLVVSHFLILHGZLWKWKH-RUHVRXUFHRSWLRQ
% bsub -n 2 -R "type==SUNSOL || type==RS6K" pam /user/batch/%a/myjob
)RUERWKWKHVHH[DPSOHVWKH3DUDOOHO$SSOLFDWLRQ0DQDJHU3$0VXEVWLWXWHVWKH
%aQRWDWLRQZLWKWKHFRUUHFW/6)KRVWW\SHSDWKQDPH7KHSDWKVXVHGWRVHOHFWWKH
ELQDULHVDUH
•
•
/user/batch/SUNSOL/myjob
/user/batch/RS6K/myjob
LSF Parallel User’s Guide 33
4
Submitting Parallel Applications
Interactive Execution
Interactive Execution
7KH/6)3DUDOOHOV\VWHPXVHVWKH3DUDOOHO$SSOLFDWLRQ0DQDJHU3$0WRFRQWURO
WKHH[HFXWLRQRISDUDOOHOEDWFKMREVLQWHUDFWLYHO\%DWFKMREVDUHH[HFXWHG
LQWHUDFWLYHO\XVLQJWKHpamFRPPDQGVHH7KHSDP&RPPDQGRQSDJH:KHQ
VXEPLWWLQJEDWFKMREVXVLQJWKHpamFRPPDQGWKH/6)%DWFKV\VWHPLVE\SDVVHG
WKHMREVDUHQRWTXHXHG%DWFKMREVDUHUXQLPPHGLDWHO\XSRQHQWHULQJWKH
FRPPDQGLIWKHUHVRXUFHUHTXLUHPHQWVVSHFLILHGDUHPHW,IWKHUHVRXUFHVDUHQRW
DYDLODEOHWKHMRELVQRWUXQ6LQFHWKHMREVGRQRWZDLWLQWHUDFWLYHMREH[HFXWLRQLV
EHQHILFLDOIRUGHEXJJLQJSDUDOOHODSSOLFDWLRQV
7RVXFFHVVIXOO\H[HFXWHDQLQWHUDFWLYHSDUDOOHOEDWFKMREWKHpamFRPPDQGPXVW
EHUHLVVXHGDWDWLPHZKHQWKHUHVRXUFHVDUHDYDLODEOH,IVSHFLILFUHVRXUFHVDUHQRW
UHTXHVWHGWKH/6)3DUDOOHOV\VWHPZLOOUXQWKHEDWFKMRERQWKHOHDVWORDGHGKRVWV
WKDWPHHWWKHEDWFKMREVFULWHULD
'LUHFWLQWHUDFWLRQLVVXSSRUWHG$OOWKHLQSXWDQGRXWSXWLVKDQGOHGWUDQVSDUHQWO\
EHWZHHQORFDODQGH[HFXWLRQKRVWV$OOMREFRQWUROVLJQDOVHJFWUO[FWUO]DQG
FWUOODUHSURSDJDWHGWRWKHH[HFXWLRQKRVWVWKLVDOORZVLQWHUDFWLRQZLWKWKHMRE
DVLILWZHUHDEHLQJH[HFXWHGORFDOO\
7KLVSDUWRIWKHFKDSWHUGLVFXVVHVWKHIROORZLQJWRSLFV
7KHSDP&RPPDQG 3URFHVV6WDWXV5HSRUW
*HWWLQJ+RVW,QIRUPDWLRQ
34 LSF Parallel User’s Guide
Submitting Parallel Applications
The pam Command
The pam Command
7KHpamFRPPDQGLVXVHGWRLQWHUDFWLYHO\H[HFXWHSDUDOOHOEDWFKMREVLQWKH/6)
3DUDOOHOV\VWHP$VXEVHWRIWKHpamFRPPDQGLVXVHGDVDFRPPDQGRSWLRQIRU
WKHbsubFRPPDQGVHH7KHEVXE&RPPDQGRQSDJH7KHV\QWD[IRUXVLQJWKH
pamFRPPDQGLV
pam
[-h][-V][-i][-t][-v]
[-server_addr location
]
[| -server_jobid location
]
[| -server_jobname location ]
}
{-m “host ...”
{| [-R req] -n num }
job [arg ...]
2SWLRQ
K
9
L
'HVFULSWLRQ
3ULQWFRPPDQGXVDJHWRVWDQGDUGHUURUDQGH[LW
3ULQW/6)YHUVLRQWRVWDQGDUGHUURUDQGH[LW
6SHFLILHVLQWHUDFWLYHRSHUDWLRQPRGHWKHXVHUZLOOEHDVNHG
LIDSSOLFDWLRQLVWREHH[HFXWHGRQDOOKRVWV
,I\HV\WKHWDVNLVVWDUWHGRQDOOKRVWVVSHFLILHGLQWKHOLVW
W
Y
VHUYHUBDGGU
ORFDWLRQ
VHUYHUBMRELG
ORFDWLRQ
VHUYHUBMREQDPH
ORFDWLRQ
,IQRQWKHXVHUPXVWLQWHUDFWLYHO\VSHFLI\WKHKRVWV
6XSSUHVVWKHSULQWLQJRIWKHMREWDVNVXPPDU\UHSRUWWRWKH
VWDQGDUGRXWSXWDWMREFRPSOHWLRQ
6SHFLILHVWKHMRELVWREHUXQLQYHUERVHPRGH7KHQDPHVRI
WKHVHOHFWHGKRVWVDUHGLVSOD\HG
6SHFLILHVWKHORFDWLRQRIWKH3$0VHUYHU7KHORFDWLRQLV
VSHFLILHGLQWKHhostname:port_noIRUPDW
6SHFLILHVWKHORFDWLRQRIWKH3$0VHUYHU7KHORFDWLRQLV
VSHFLILHGXVLQJWKHjobidIRUWKHVHUYHU3$0MRE
6SHFLILHVWKHORFDWLRQRIWKH3$0VHUYHU7KHORFDWLRQLV
VSHFLILHGXVLQJWKHjobnameIRUWKHVHUYHU3$0MRE
LSF Parallel User’s Guide 35
4
Submitting Parallel Applications
The pam Command
2SWLRQ
PKRVW«
'HVFULSWLRQ
6SHFLILHVWKHOLVWRIKRVWVRQZKLFKWRUXQWKHSDUDOOHOEDWFK
MREWDVNV7KHQXPEHURIKRVWQDPHVVSHFLILHGLQGLFDWHVWKH
QXPEHURISURFHVVRUVUHTXHVWHG
7KLVRSWLRQFDQQRWEHXVHGZLWKRSWLRQV-RRU-n
DQGLVLJQRUHGZKHQpamLVXVHGDVDbsubRSWLRQ
6SHFLILHVWKHQXPEHURISURFHVVRUVUHTXLUHGWRUXQWKH
SDUDOOHOMRE
Note
>5UHT@QQXP
7KLVRSWLRQFDQQRWEHXVHGZLWKRSWLRQ-mDQGLV
LJQRUHGZKHQpamLVXVHGDVDbsubRSWLRQ
6SHFLI\WKHUHVRXUFHUHTXLUHPHQWVIRUKRVWVHOHFWLRQ([HFXWH
WKHSDUDOOHOMRERQWKHKRVWVWKDWPHHWWKHVHUHTXLUHPHQWV
Note
5UHT
'HIDXOWr15s:pg
7KLVRSWLRQLVLJQRUHGZKHQpamLVXVHGDVDbsub
RSWLRQ
7KHQDPHRIWKHSDUDOOHOMREWREHUXQ
Note
MRE>DUJ«@
Note
OLQH
7KLVPXVWEHWKHODVWDUJXPHQWRQWKHSDPFRPPDQG
([DPSOH7KHIROORZLQJFRPPDQGH[HFXWHVWKHSDUDOOHOEDWFKMREQDPHGmyjob
RQWKH/6)3DUDOOHOV\VWHPUHTXHVWLQJIRXUSURFHVVRUVRIDQ\W\SH
% pam -n 4 myjob
TID HOST_NAME
COMMAND_LINE
==== ========== ================
1
host1
myjob
2
host2
myjob
3
host3
myjob
4
host4
myjob
36 LSF Parallel User’s Guide
STATUS
========================
Completed
Completed
Completed
Completed
TERMINATION_TIME
==================
03/31/98 10:31:58
03/31/98 10:31:59
03/31/98 10:31:59
03/31/98 10:31:58
Submitting Parallel Applications
The pam Command
([DPSOH7KHIROORZLQJFRPPDQGXVHVWKH-mRSWLRQWRH[HFXWHWKHSDUDOOHOEDWFK
MREQDPHGmyjobRQKRVWKRVWDQGKRVW
% pam -m "host1
TID HOST_NAME
==== ==========
1
host1
2
host2
3
host3
host2 host3" myjob
COMMAND_LINE
STATUS
================ ========================
myjob
Completed
myjob
Completed
myjob
Completed
TERMINATION_TIME
==================
03/31/98 10:31:58
03/31/98 10:31:59
03/31/98 10:31:59
LSF Parallel User’s Guide 37
4
Submitting Parallel Applications
Process Status Report
Process Status Report
$IWHUDSDUDOOHOEDWFKMREWHUPLQDWHVLQDVXFFHVVIXO'RQHRUIDLOHG(;,7VWDWH
WKH/6)3DUDOOHOV\VWHPGLVSOD\VWKHVWDWXVRIDOOWKHSURFHVVHV)RUH[DPSOH
% pam -n 4 myjob
TID HOST_NAME
COMMAND_LINE
==== ========== ================
1
host1
myjob
2
host2
myjob
3
host3
myjob
3
host4
myjob
STATUS
========================
Done
Done
Done
Done
TERMINATION_TIME
==================
03/31/98 10:31:58
03/31/98 10:31:59
03/31/98 10:31:59
03/31/98 10:31:59
7DEOHpam -n-RE6WDWXV
6WDWXV
'HVFULSWLRQ
'RQH
3URFHVVVXFFHVVIXOO\FRPSOHWHGZLWKH[LWFRGHRI
([LWFRGH
3URFHVVXQVXFFHVVIXOO\FRPSOHWHGZLWKDQH[LWFRGHRI
FRGH
6LJQDOHGVLJQDO
3URFHVVZDVWHUPLQDWHGE\VLJQDO
([LWVWDWXVXQNQRZQ &RQQHFWLRQEURNHQH[LWVWDWXVXQNQRZQ
.LOOHGE\3$0VLJQDO 3$0VKXWGRZQSURFHVVXVLQJVLJQDO
8QUHDFKDEOH
3$0LVXQDEOHWRUHDFKKRVWDIWHUEURNHQFRQQHFWLRQ
1RZD\WRGHWHUPLQHWKHVWDWHRIWKHSURFHVV
5XQDZD\
3URFHVVLVVWLOOUXQQLQJFDQQRWEHNLOOHGE\3$0
6XVSHQG
3URFHVVVXVSHQGHG
8QGHILQHG
"""
5XQ
3URFHVVUXQQLQJ
/RFDO5(6GLHG
"""
Note
8VHWKH-tRSWLRQWRVXSSUHVVWKHSURFHVV6WDWXVUHSRUW)RUH[DPSOH
pam -t -n 4 myjob
38 LSF Parallel User’s Guide
Submitting Parallel Applications
Getting Host Information
Getting Host Information
7KHlshostsFRPPDQGLVXVHGWRGLVSOD\LQIRUPDWLRQDERXW/6)KRVW
FRQILJXUDWLRQVLQFOXGLQJQDPHW\SHPRGHO&38QRUPDOL]DWLRQIDFWRUQXPEHURI
&38VWRWDOPHPRU\DQGDYDLODEOHUHVRXUFHV
([DPSOH
% lshosts
HOST_NAME
host1
host99
host2
host7
host3
host6
host4
host5
host8
type
model
SGI64 SGI4D35
SUNSOL SunSparc
LINUX I486_33
SUN41 SPARCSLC
ALPHA~1 DEC5000
ALPHA~1 DEC5000
SUNSOL SunSparc
SGI SGIINDIG
SUNSOL SunSparc
cpuf ncpus maxmem maxswp server RESOURCES
2.0
1
96M
153M
Yes (lsf_js irix gla)
12.0
4 1024M 1930M
Yes (solaris cs bigmem)
14.0
1
30M
64M
Yes (linux)
3.0
1
15M
29M
Yes (sparc bsd sun41)
5.0
1
88M
384M
Yes (cs bigmem alpha gla)
5.0
1
84M
350M
Yes (gla)
12.0
2
256M
733M
Yes (solaris cs bigmem)
15.0
1
96M
300M
Yes (irix)
12.0
1
56M
90M
Yes (solaris cs bigmem)
LSF Parallel User’s Guide 39
4
Submitting Parallel Applications
Getting Host Information
40 LSF Parallel User’s Guide
AVendor MPI Implementations
LSF Parallel User’s Guide 41
HP MPI
HP MPI
:KHQ\RXXVHmpirunLQVWDQGDORQHPRGH\RXSURYLGHLWWKHKRVWQDPHVWREH
XVHGE\WKH03,MRE7RDFKLHYHEHWWHUUHVRXUFHXWLOL]DWLRQ\RXFDQKDYH/6)
PDQDJHWKHDOORFDWLRQRIKRVWVFRRUGLQDWLQJWKHVWDUWXSSKDVHZLWKmpirun7KLV
LVGRQHE\SUHFHGLQJWKHUHJXODU+303,mpirunFRPPDQGZLWK
% bsub pam -mpi
([DPSOH7RUXQDVLQJOHKRVWMREDQGKDYHWKH/6)%DWFKV\VWHPVHOHFWWKHKRVW
WKHFRPPDQG
% mpirun -np 14 a.out
LVHQWHUHGDV
% bsub pam -mpi mpirun -np 14 a.out
([DPSOH7RUXQDPXOWLKRVWMREDQGKDYHWKH/6)%DWFKV\VWHPVHOHFWWKHKRVWV
WKHFRPPDQG
% mpirun -f appfile
LVHQWHUHGDV
% bsub pam -mpi mpirun -f appfile
ZKHUHappfileFRQWDLQVWKHIROORZLQJHQWULHV
-h foo -np 8 a.out
-h bar -np 4 b.out
-h foo -np 2 c.out
,QWKLVH[DPSOHWKHKRVWVfooDQGbarDUHWUHDWHGDVV\PEROLFQDPHVDQGUHIHUWR
WKHDFWXDOKRVWVWKDWWKH/6)%DWFKV\VWHPDOORFDWHVWRWKHMRE7KHa.outDQG
c.outSURFHVVHVDUHJXDUDQWHHGWRUXQRQWKHVDPHKRVW7KHb.outSURFHVVHVPD\
UXQRQDGLIIHUHQWKRVWGHSHQGLQJRQWKHUHVRXUFHVDYDLODEOHDQGWKH/6)%DWFK
V\VWHPVFKHGXOLQJDOJRULWKPV
)RUDFRPSOHWHOLVWRImpirunRSWLRQVDQGHQYLURQPHQWYDULDEOHFRQWUROVUHIHUWR
WKHmpirunPDQSDJHDQGWKH+303,8VHU
V*XLGHYHUVLRQ
42 LSF Parallel User’s Guide
SGI MPI
SGI MPI
7KH-mpiDUJXPHQWRQWKHbsubDQGpamFRPPDQGOLQHLVDUHSODFHPHQWIRU
mpirunLQWKH+3HQYLURQPHQW(YHU\WKLQJDIWHU-mpiVKDOOEHH[DFWO\DVLWZRXOG
QRUPDOO\DSSHDULImpirunZHUHEHLQJXVHG
([DPSOH7RUXQDWKHa.outMREDQGKDYHWKH/6)%DWFKV\VWHPVHOHFWWKHKRVW
WKHFRPPDQG
% mpirun -np 4 a.out
LVHQWHUHGDV
% mpirun pam -mpi -np 4 a.out
([DPSOH7RUXQDPXOWLKRVWMREDQGKDYHWKH/6)%DWFKV\VWHPVHOHFWWKHKRVWV
WKHIROORZLQJFRPPDQG
% mpirun -f appfile
LVHQWHUHGDV
% bsub pam -mpi -f appfile
ZKHUHDSSILOHFRQWDLQVWKHIROORZLQJHQWULHV
foo -np 4 a.out
bar -np 4 b.out
foo -np 2 c.out
)RUDFRPSOHWHOLVWRImpirunRSWLRQVDQGHQYLURQPHQWYDULDEOHFRQWUROVUHIHUWR
WKHmpirunPDQSDJH
LSF Parallel User’s Guide 43
SUN HPC MPI
SUN HPC MPI
:KHQUXQQLQJ/6)%DWFKMREVRQ6XQSODWIRUPV\RXFDQLQFOXGHWKH6XQVSHFLILF
DUJXPHQW²sunhpcRQWKHbsubFRPPDQGOLQHDIWHUDQ\RWKHUbsubDUJXPHQWV
7KHIROORZLQJDUJXPHQWVWR²sunhpcSURYLGHDGGLWLRQDOFRQWURORYHUbsub
EHKDYLRULQD6XQ+3&HQYLURQPHQW
–n processes
6SHFLI\WKHQXPEHURISURFHVVHVWRUXQ1RWHWKDWWKHbsub –nDUJXPHQW
VSHFLILHVWKHQXPEHURI&38VWREHXVHGIRUWKHMRE
([DPSOH7RVWDUWDSURFHVVLQWHUDFWLYHMRERQ3$0HQDEOHGTXHXHhpc
WKDWZLOOZUDSRYHUDWOHDVWDQGDVPDQ\DV&38V
% bsub -I -n 4,16 -q hpc -sunhpc -n 48 jobname
6HWWLQJWKHPLQLPXPQXPEHURI&38VWRDQXPEHUJUHDWHUWKDQUDLVHV
WKHSRVVLELOLW\WKDWLIWKHUHDUHIHZHU&38VDYDLODEOHWKDQWKHPLQLPXP
QXPEHU\RXVSHFLI\WKHMREPD\IDLOWRVWDUW,QWKLVH[DPSOHLIIHZHUWKDQ
&38VDUHDYDLODEOHWKHMREZLOOQRWVWDUW<RXFDQDYRLGWKLVSRWHQWLDOSUREOHP
E\VHWWLQJWKHPLQLPXPQXPEHURI&38VWR+RZHYHUWKLVLQWURGXFHVWKH
SRWHQWLDOFRVWWRSHUIRUPDQFHRIKDYLQJWKHSURFHVVHVZUDSSHGRYHUDVPDOOHU
QXPEHURI&38V
Note
–P host:port
6SHFLI\WKH3$0DGGUHVVRIDQRWKHUMREZLWKZKLFKWKHQHZMREVKRXOG
FRORFDWH7KH3$0DGGUHVVLVWKH7&3VRFNHWXVHGIRUFRPPXQLFDWLRQV
EHWZHHQWKHMREDQG3$0
([DPSOH7RVWDUWD&38LQWHUDFWLYHMRERQ3$0HQDEOHGTXHXHhpc
bsub
–I –n 4 –q hpc –sunhpc –PAthos:123MREQDPH
7KHQHZMRELVFRORFDWHGZLWKWKHMREZKRVH3$0LVUXQQLQJRQKRVW$WKRV
XVLQJSRUW
–j job_ID
6SHFLI\WKHMRE,'RIDQRWKHUMREZLWKZKLFKWKHQHZMREVKRXOGFRORFDWH
–J job_name
6SHFLI\WKHMREQDPHRIDQRWKHUMREZLWKZKLFKWKHQHZMREVKRXOGFRORFDWH
44 LSF Parallel User’s Guide
SUN HPC MPI
–s
6SHFLI\WKDWWKHMRELVWREHVSDZQHGLQWKH67233('VWDWH
7RLGHQWLI\SURFHVVHVLQWKH67233('VWDWHLVVXHWKHpsFRPPDQGZLWKWKH
–elDUJXPHQW
orpheus 215 => ps -el
F S
UID
PID PPID
19 T
0
0
0
C PRI NI
0 0
SY
ADDR
f0274e38
SZ
0
WCHAN TTY
?
TIME CMD
0:00 sched
+HUHWKHschedFRPPDQGLVLQ67233('VWDWHDVLQGLFDWHGE\WKHTHQWU\
LQWKHS6WDWHFROXPQ
1RWHWKDWZKHQVSDZQLQJDSURFHVVLQWKH67233('VWDWHXQGHU/6)WKH
QDPHRI\RXUSURJUDPZLOOQRWDSSHDULQWKHpsRXWSXW,QVWHDGWKHVWRSSHG
SURFHVVZLOOEHLGHQWLILHGDVD5(6GDHPRQ
([DPSOH7RVWDUWD&38LQWHUDFWLYHMRERQ3$0HQDEOHGTXHXHhpcLQWKH
67233('VWDWH
% bsub –I –n 1 –q hpc –sunhpc –s jobname
LSF Parallel User’s Guide 45
SUN HPC MPI
46 LSF Parallel User’s Guide
Index
D
DONE . . . . . . . . . . . . . . . . . . . . . . . . 24
Symbols
E
%a notation . . . . . . . . . . . . . . . . . . . . 18
execution host substitution. . . . . . . 18
EXIT . . . . . . . . . . . . . . . . . . . . . . . . . . 25
B
F
batch job
interactive execution . . . . . . . . . 35
monitor . . . . . . . . . . . . . . . . . . . . 30
resource usage . . . . . . . . . . . . . . 30
resume . . . . . . . . . . . . . . . . . . . . . 29
submit . . . . . . . . . . . . . . . . . . . . . 26
suspend . . . . . . . . . . . . . . . . . . . . 28
terminate . . . . . . . . . . . . . . . . . . . 31
batch job state . . . . . . . . . . . . . . . . . . 23
batch job status . . . . . . . . . . . . . . . . . 23
bjobs . . . . . . . . . . . . . . . . . . . . . . . . . . 30
bkill. . . . . . . . . . . . . . . . . . . . . . . . . . . 31
bresume . . . . . . . . . . . . . . . . . . . . . . . 29
bstop. . . . . . . . . . . . . . . . . . . . . . . . . . 28
bsub . . . . . . . . . . . . . . . . . . . . . . . . . . 26
pam option . . . . . . . . . . . . . . . . . 26
C
C program
compile . . . . . . . . . . . . . . . . . . . . 15
command syntax . . . . . . . . . . . . . . xii
compile . . . . . . . . . . . . . . . . . . . . . . . 15
C program . . . . . . . . . . . . . . . . . . 15
Fortran 77 program . . . . . . . . . . 15
Fortran 77 program
compile . . . . . . . . . . . . . . . . . . . . 15
H
host type substitution . . . . . . . . . . . 18
HP MPI . . . . . . . . . . . . . . . . . . . . . . . 42
I
interactive execution
batch job . . . . . . . . . . . . . . . . . . . 35
J
job
interactive execution . . . . . . . . . 35
monitor . . . . . . . . . . . . . . . . . . . . 30
resource usage . . . . . . . . . . . . . . 30
resume . . . . . . . . . . . . . . . . . . . . . 29
submit . . . . . . . . . . . . . . . . . . . . . 26
suspend . . . . . . . . . . . . . . . . . . . . 28
terminate . . . . . . . . . . . . . . . . . . . 31
job state . . . . . . . . . . . . . . . . . . . . . . . 23
DONE . . . . . . . . . . . . . . . . . . . . . 24
EXIT . . . . . . . . . . . . . . . . . . . . . . . 25
LSF Parallel User’s Guide 47
PEND . . . . . . . . . . . . . . . . . . . . . . 24
PSUSP . . . . . . . . . . . . . . . . . . . . . 24
RUN . . . . . . . . . . . . . . . . . . . . . . . 24
SSUSP. . . . . . . . . . . . . . . . . . . . . . 25
USUSP . . . . . . . . . . . . . . . . . . . . . 24
job status . . . . . . . . . . . . . . . . . . . . . . 23
pam option
%a . . . . . . . . . . . . . . . . . . . . . . . . . 18
PEND . . . . . . . . . . . . . . . . . . . . . . . . . 24
PSUSP . . . . . . . . . . . . . . . . . . . . . . . . 24
L
resource usage
batch job . . . . . . . . . . . . . . . . . . . 30
resume
batch job . . . . . . . . . . . . . . . . . . . 29
RUN . . . . . . . . . . . . . . . . . . . . . . . . . . 24
link . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
C program . . . . . . . . . . . . . . . . . . 15
Fortran 77 program . . . . . . . . . . 15
lshosts command . . . . . . . . . . . . . . . 39
R
S
M
monitor
batch job. . . . . . . . . . . . . . . . . . . . 30
MPI
HP. . . . . . . . . . . . . . . . . . . . . . . . . 42
SGI . . . . . . . . . . . . . . . . . . . . . . . . 43
SUN HPC . . . . . . . . . . . . . . . . . . 44
mpicc . . . . . . . . . . . . . . . . . . . . . . . . . 15
mpif77 . . . . . . . . . . . . . . . . . . . . . . . . 15
SGI MPI . . . . . . . . . . . . . . . . . . . . . . . 43
SSUSP. . . . . . . . . . . . . . . . . . . . . . . . . 25
submit
batch job . . . . . . . . . . . . . . . . . . . 26
substitution
host type . . . . . . . . . . . . . . . . . . . 18
SUN HPC MPI . . . . . . . . . . . . . . . . . 44
suspend
batch job . . . . . . . . . . . . . . . . . . . 28
syntax. . . . . . . . . . . . . . . . . . . . . . . . xii
N
T
notation
%a . . . . . . . . . . . . . . . . . . . . . . . . . 18
P
pam
bsub option . . . . . . . . . . . . . . . . . 26
command. . . . . . . . . . . . . . . . . . . 35
48 LSF Parallel User’s Guide
terminate
batch job . . . . . . . . . . . . . . . . . . . 31
typographic conventions . . . . . . . . xi
U
USUSP . . . . . . . . . . . . . . . . . . . . . . . . 24