(→Example Job) |
|||
| Line 108: | Line 108: | ||
</pre> | </pre> | ||
| − | =General Notes= | + | ==General Notes== |
| − | ==Checking Job Status== | + | ===Checking Job Status=== |
To check the status of your job in the queue use the sqjobs command. On x86_64 systems you can check whether any of your jobs are queued on the license server use <i>lstc_qrun</i>. Its possible that sqsub will start your job but it will sit idle until enough license are available on the license server and the lstc_qrun command will reveal this: | To check the status of your job in the queue use the sqjobs command. On x86_64 systems you can check whether any of your jobs are queued on the license server use <i>lstc_qrun</i>. Its possible that sqsub will start your job but it will sit idle until enough license are available on the license server and the lstc_qrun command will reveal this: | ||
| Line 127: | Line 127: | ||
</source> | </source> | ||
| − | ==Checking License Details== | + | ===Checking License Details=== |
At the time of this writing UWO is running on a 4cpu demo license as follows: | At the time of this writing UWO is running on a 4cpu demo license as follows: | ||
| Line 177: | Line 177: | ||
Note! the <i>ls971_s_R3_1</i> and <i>ls971_d_R3_1</i> binaries do not work, a fix is being looked for. | Note! the <i>ls971_s_R3_1</i> and <i>ls971_d_R3_1</i> binaries do not work, a fix is being looked for. | ||
| − | + | =References= | |
o LSTC LS-DYNA Homepage<br> | o LSTC LS-DYNA Homepage<br> | ||
http://www.lstc.com/lsdyna.htm | http://www.lstc.com/lsdyna.htm | ||
Revision as of 15:51, 10 April 2013
| LS-DYNA |
|---|
| Description: Suite of programs for transient dynamic finite element program |
| SHARCNET Package information: see LS-DYNA software page in web portal |
| Full list of SHARCNET supported software |
Contents
Introduction
Researchers who want to use LSDYNA must either purchase a license for the sharcnet license server OR have both sharcnet computing hardware at their local institution (mac, uwo, guelph, waterloo) and an accessible institutional license server.
Usage
These usage instructions apply to the module based installation of lsdyna found under directory /opt/sharcnet/lsdyna on licensed clusters shown in the above Availability Table. To access and use this software you must be a member of the lsdyna group. Users who belong to a licensed department will be given a 5 digit port number. The value should then be inserted into the appropriate departmental export statement before loading the module file as follows:
o UWO Civil and Environmental Engineering Dept
export LSTC_LICENSE_SERVER='PortNumber'@license1.uwo.sharcnet
o McGill Mechanical Engineering Department
export LSTC_LICENSE_SERVER='PortNumber'@license2.uwo.sharcnet
o UW Mechanical and Mechatronics Engineering Dept
export LSTC_LICENSE_SERVER='PortNumber'@license3.uwo.sharcnet
Module Selection
[roberpj@orc-login1:~] module avail lsdyna -------------------------------------- /opt/sharcnet/modules --------------------------------------- lsdyna/ls971mpp6.0.0 lsdyna/ls971mpp6.1.1 lsdyna/ls971smp6.1.0 lsdyna/ls980mppB1 lsdyna/ls971mpp6.1.0 lsdyna/ls971smp6.0.0 lsdyna/ls971smp6.1.1 lsdyna/ls980smpB1
o For serial or threaded jobs (using ls971 version 6.0.0 or 6.1.0 or 6.1.1) do:
module load lsdyna/ls971smp6.1.1
o For mpi jobs (using ls971 version 6.0.0 or 6.1.0 or 6.1.1) do:
module unload intel openmpi lsdyna module load intel/11.1.069 openmpi/intel/1.5.4 lsdyna/ls971mpp6.1.1
o For serial or threaded jobs (using version ls980) do:
module load lsdyna/ls980smpB1
o For mpi jobs (using version ls980) do:
module unload intel openmpi lsdyna module load intel/12.1.3 openmpi/intel/1.4.5 lsdyna/ls980mppB1
Job Submission
To run the single or double precision solvers specify lsdyna_s or lsdyna_d respectively:
o SUBMIT 1CPU SERIAL JOB
sqsub -r 7d -q serial -o ofile.%J --mpp=2G lsdyna_d i=airbag.deploy.k ncpu=1
o SUBMIT 4CPU SMP JOB
sqsub -r 7d -q threaded -n 4 -o ofile.%J --mpp=2G lsdyna_d i=airbag.deploy.k ncpu=4
o SUBMIT 4CPU MPI JOB
sqsub -r 7d -q mpi -n 4 -o ofile.%J --mpp=1G lsdyna_d i=airbag.deploy.k
Example Job
STEP1) The following shows a sample sqsub jobs submission for the airbag example to the mpi queue as shown in this link http://www.dynaexamples.com/examples-manual/misc/airbag:
cp -a /opt/sharcnet/lsdyna/ls971smp6.1.0/examples /work/$USER/test-lsdyna cd /work/$USER/test-lsdyna/misc/airbag gunzip airbag.deploy.k.gz sqsub -r 1h -q mpi --mpp=4G -n 8 -o ofile.%J lsdyna_d i=airbag.deploy.k
STEP2) With the job still running, use the following echo command to trigger generation of restart files at which point the file D3KIL will vanish:
echo "sw3" > D3KIL
STEP3) Now either manually kill off the job or wait until it exceeds the -r time limit:
sqkill job#
STEP4) Before the job can be restarted, the following two lines must be added to the airbag.deploy.restart.k file:
*CHANGE_CURVE_DEFINITION
1
STEP5) Now resubmit the job as follows using "r=" to specify the restart file:
sqsub -r 1h -q mpi --mpp=4G -n 8 -o ofile.%J lsdyna_d i=airbag.deploy.restart.k r=d3dump01
General Notes
Checking Job Status
To check the status of your job in the queue use the sqjobs command. On x86_64 systems you can check whether any of your jobs are queued on the license server use lstc_qrun. Its possible that sqsub will start your job but it will sit idle until enough license are available on the license server and the lstc_qrun command will reveal this:
[roberpj@hnd19:~/samples/lsdyna] lstc_qrun Defaulting to server 1 specified by LSTC_LICENSE_SERVER variable Running Programs User Host Program Started # procs ----------------------------------------------------------------------------- hsnmoham 12208@orc271.orca.shar MPPDYNA Mon Jan 7 18:17 24 roberpj 9767@hnd15 LS-DYNA_971 Mon Jan 7 19:42 4 No programs queued
Checking License Details
At the time of this writing UWO is running on a 4cpu demo license as follows:
[roberpj@hnd19:~/samples/lsdyna] lstc_qrun -R Defaulting to server 1 specified by LSTC_LICENSE_SERVER variable LICENSE INFORMATION PROGRAM EXPIRATION CPUS USED FREE MAX | QUEUE ---------------- ---------- ----- ------ ------ | ----- LS-DYNA_971 05/31/2013 - 172 200 | 0 roberpj 9767@hnd15 4 MPPDYNA_971 05/31/2013 0 172 200 | 0 LS-DYNA 05/31/2013 0 172 200 | 0 MPPDYNA 05/31/2013 - 172 200 | 0 hsnmoham 12208@orc271.orca.sharcnet 24 LICENSE GROUP 28 172 200 | 0 PROGRAM EXPIRATION CPUS USED FREE MAX | QUEUE ---------------- ---------- ----- ------ ------ | ----- LS-OPT 05/31/2013 0 200 200 | 0 LICENSE GROUP 0 200 200 | 0
Submitting Jobs Using Legacy Versions
The following binaries remain available on saw and orca for backward compatibility testing:
[roberpj@orc-login2:~] cd /opt/sharcnet/local/lsdyna [roberpj@orc129:/opt/sharcnet/local/lsdyna] ls ls971* ls971_d_R3_1 ls971_d_R4_2_1 ls971_s_R3_1 ls971_s_R4_2_1 ls971_s_R5_1_1 ls971_d_R4_2_0 ls971_d_R5_0 ls971_s_R4_2_0 ls971_s_R5_0
There are currently no sharcnet modules for these versions, hence jobs should be submitted as follows:
module load lsdyna export PATH=/opt/sharcnet/local/lsdyna:$PATH export LSTC_LICENSE_SERVER=XXXXX@license3.uwo.sharcnet cp /opt/sharcnet/local/lsdyna/examples/airbag.deploy.k airbag.deploy.k SERIAL JOB: sqsub -q serial -r 1d -o ofile.%J ls971_d_R4_2_1 i=airbag.deploy.k THREADED JOB: sqsub -q threaded -n 4 -r 1d -o ofile.%J ls971_d_R4_2_1 ncpu=4 para=2 i=airbag.deploy.k
Note! the ls971_s_R3_1 and ls971_d_R3_1 binaries do not work, a fix is being looked for.
References
o LSTC LS-DYNA Homepage
http://www.lstc.com/lsdyna.htm