| Modifier and Type | Field and Description |
|---|---|
protected static String |
CLUSTER_BATCH_COMMAND
Config String property: "cluster.batchCommand" |
protected static String |
CLUSTER_HAS_BATCH_IDS
Config Boolean property: "cluster.returnsBatchIds" |
protected static String |
CLUSTER_MODULES
Config List property: "cluster.modules" |
protected static List<String> |
CLUSTER_NUM_PROCESSORS
One parameter of the
Config String property "cluster.jobHeader" to set number of cores. |
protected static String |
CLUSTER_PROLOGUE
Config String property: "cluster.prologue" |
protected static String |
CLUSTER_STATUS_COMMAND
Config String property: "cluster.statusCommand" |
protected static String |
SCRIPT_JOB_HEADER
Config String property: "cluster.jobHeader" |
| Modifier and Type | Method and Description |
|---|---|
protected static List<String> |
buildExecuteFunction()
Create bash worker script function: executeLine
Capture status code of the payload script line. |
protected static void |
buildMainScript(ScriptModule module)
Build the MIAN script.
|
protected static List<String> |
buildScriptFailureFunction(String path)
Create bash worker script function: scriptFailed
Failure details written to the failure indicator file: $1 Script payload line to execute $2 Script line number $3 Script line failure status code |
static void |
buildScripts(ScriptModule module,
List<List<String>> data)
This method builds the bash scripts required for the given module.
Standard local/cluster pipelines include: 1 MAIN script, 1+ worker-scripts. Docker R_Modules include: 1 MAIN script, 0 worker-scripts - MAIN.R run by MAIN.sh Docker *non-R_Modules* include: 1 MAIN script, 1+ worker-scripts - MAIN.sh runs workers AWS Docker R_Modules include: 0 MAIN scripts, 0 worker-scripts - MAIN.R run by Nextflow AWS Docker *non-R_Modules* include: 0 MAIN scripts, 1+ worker-scripts MAIN.sh runs workers |
static void |
checkDependencies(BioModule module) |
protected static File |
createScript(ScriptModule module,
String scriptPath,
List<String> lines)
Create the script.
|
protected static String |
getMainScriptExecuteWorkerLine(String workerScriptPath)
Call executeLine function in the worker script
|
protected static List<String> |
getWorkerScriptLines(List<String> lines)
Pass each line and the current line number to: executeLine
|
protected static String |
getWorkerScriptPath(ScriptModule module)
Build the file path for the numbered worker script.
|
protected static List<String> |
initMainScript(ScriptModule module)
Create the ScriptModule main script that calls all worker scripts.
|
protected static List<String> |
initWorkerScript(ScriptModule module,
String scriptPath)
Create the numbered worker scripts.
|
protected static List<String> |
insertPathVars(ScriptModule module,
List<String> scriptLines)
Paths to common module directories are added via
pathVariableVals. |
static List<String> |
listProps()
Let modules see property names.
|
static List<String> |
loadModules(ScriptModule module)
Return lines to script that load cluster modules based on
Config."cluster.modules" |
protected static List<String> |
passEnvironmentVarVals(ScriptModule module) |
protected static List<String> |
pathVariableVals(ScriptModule module)
Get the script lines to assign values to the common directories.
|
static void |
registerProps()
Register properties with the Properties class for API access.
|
protected static void |
writeScript(ScriptModule module,
BufferedWriter writer,
List<String> scriptLines)
This method formats the bash script to indent if statement code blocks
|
protected static final String CLUSTER_BATCH_COMMAND
Config String property: "cluster.batchCommand"protected static final String CLUSTER_HAS_BATCH_IDS
Config Boolean property: "cluster.returnsBatchIds"protected static final String CLUSTER_STATUS_COMMAND
Config String property: "cluster.statusCommand"protected static final String CLUSTER_MODULES
Config List property: "cluster.modules"protected static final List<String> CLUSTER_NUM_PROCESSORS
Config String property "cluster.jobHeader" to set number of cores.protected static final String CLUSTER_PROLOGUE
Config String property: "cluster.prologue"protected static final String SCRIPT_JOB_HEADER
Config String property: "cluster.jobHeader"public static void buildScripts(ScriptModule module, List<List<String>> data) throws PipelineScriptException
module - ScriptModuledata - Bash script linesPipelineScriptException - if any errors occur writing module scriptprotected static List<String> buildExecuteFunction()
protected static void buildMainScript(ScriptModule module) throws ConfigException, IOException, DockerVolCreationException
module - ScriptModuleIOException - if errors occur writing the MAIN script linesConfigException - if problems occur accessing Config propertiesDockerVolCreationExceptionprotected static List<String> buildScriptFailureFunction(String path)
path - Script pathprotected static File createScript(ScriptModule module, String scriptPath, List<String> lines) throws IOException
module - ScriptModulescriptPath - Worker script pathlines - Ordered bash script lines to write to the scriptPathIOException - if unable to write the bash script to file systemprotected static String getMainScriptExecuteWorkerLine(String workerScriptPath)
workerScriptPath - Worker script pathprotected static List<String> getWorkerScriptLines(List<String> lines)
lines - Basic script lines generated by a BioModule.protected static String getWorkerScriptPath(ScriptModule module) throws ConfigNotFoundException, ConfigFormatException
module - ScriptModule is the module that owns the scriptsConfigFormatException - if "script.numWorkers" property is not a positive
integerConfigNotFoundException - if "script.numWorkers" property is undefinedprotected static List<String> initMainScript(ScriptModule module) throws ConfigException, DockerVolCreationException
module - ScriptModule script directory is where main script is writtenConfigException - if errors occur accessing Config propertiesDockerVolCreationExceptionprotected static List<String> pathVariableVals(ScriptModule module)
insertPathVars.module - insertPathVarsprotected static List<String> passEnvironmentVarVals(ScriptModule module) throws ConfigNotFoundException, ConfigFormatException
protected static List<String> insertPathVars(ScriptModule module, List<String> scriptLines)
pathVariableVals.
Using these variables in place of full file paths makes the script easier to read.module - scriptLines - pathVariableValsprotected static List<String> initWorkerScript(ScriptModule module, String scriptPath) throws Exception
module - ScriptModule script directory is where main script is writtenscriptPath - Worker script absolute file pathException - if required properties are undefinedprotected static void writeScript(ScriptModule module, BufferedWriter writer, List<String> scriptLines) throws IOException
module - ScriptModulewriter - BufferedWriter writes to the filescriptLines - List or shell script linesIOException - if errors occur writing the new scriptpublic static List<String> loadModules(ScriptModule module)
Config."cluster.modules"module - ScriptModulepublic static void registerProps()
throws API_Exception
API_Exceptionpublic static void checkDependencies(BioModule module) throws ConfigNotFoundException, ConfigFormatException