Modifier and Type | Field and Description |
---|---|
protected static String |
CLUSTER_BATCH_COMMAND
Config String property: "cluster.batchCommand" |
protected static String |
CLUSTER_HAS_BATCH_IDS
Config Boolean property: "cluster.returnsBatchIds" |
protected static String |
CLUSTER_MODULES
Config List property: "cluster.modules" |
protected static List<String> |
CLUSTER_NUM_PROCESSORS
One parameter of the
Config String property "cluster.jobHeader" to set number of cores. |
protected static String |
CLUSTER_PROLOGUE
Config String property: "cluster.prologue" |
protected static String |
CLUSTER_STATUS_COMMAND
Config String property: "cluster.statusCommand" |
protected static String |
SCRIPT_JOB_HEADER
Config String property: "cluster.jobHeader" |
Modifier and Type | Method and Description |
---|---|
protected static List<String> |
buildExecuteFunction()
Create bash worker script function: executeLine
Capture status code of the payload script line. |
protected static void |
buildMainScript(ScriptModule module)
Build the MIAN script.
|
protected static List<String> |
buildScriptFailureFunction(String path)
Create bash worker script function: scriptFailed
Failure details written to the failure indicator file: $1 Script payload line to execute $2 Script line number $3 Script line failure status code |
static void |
buildScripts(ScriptModule module,
List<List<String>> data)
This method builds the bash scripts required for the given module.
Standard local/cluster pipelines include: 1 MAIN script, 1+ worker-scripts. Docker R_Modules include: 1 MAIN script, 0 worker-scripts - MAIN.R run by MAIN.sh Docker *non-R_Modules* include: 1 MAIN script, 1+ worker-scripts - MAIN.sh runs workers AWS Docker R_Modules include: 0 MAIN scripts, 0 worker-scripts - MAIN.R run by Nextflow AWS Docker *non-R_Modules* include: 0 MAIN scripts, 1+ worker-scripts MAIN.sh runs workers |
static void |
checkDependencies(BioModule module) |
protected static File |
createScript(ScriptModule module,
String scriptPath,
List<String> lines)
Create the script.
|
protected static String |
getMainScriptExecuteWorkerLine(String workerScriptPath)
Call executeLine function in the worker script
|
protected static List<String> |
getWorkerScriptLines(List<String> lines)
Pass each line and the current line number to: executeLine
|
protected static String |
getWorkerScriptPath(ScriptModule module)
Build the file path for the numbered worker script.
|
protected static List<String> |
initMainScript(ScriptModule module)
Create the ScriptModule main script that calls all worker scripts.
|
protected static List<String> |
initWorkerScript(ScriptModule module,
String scriptPath)
Create the numbered worker scripts.
|
protected static List<String> |
insertPathVars(ScriptModule module,
List<String> scriptLines)
Paths to common module directories are added via
pathVariableVals . |
static List<String> |
listProps()
Let modules see property names.
|
static List<String> |
loadModules(ScriptModule module)
Return lines to script that load cluster modules based on
Config ."cluster.modules" |
protected static List<String> |
passEnvironmentVarVals(ScriptModule module) |
protected static List<String> |
pathVariableVals(ScriptModule module)
Get the script lines to assign values to the common directories.
|
static void |
registerProps()
Register properties with the Properties class for API access.
|
protected static void |
writeScript(ScriptModule module,
BufferedWriter writer,
List<String> scriptLines)
This method formats the bash script to indent if statement code blocks
|
protected static final String CLUSTER_BATCH_COMMAND
Config
String property: "cluster.batchCommand"protected static final String CLUSTER_HAS_BATCH_IDS
Config
Boolean property: "cluster.returnsBatchIds"protected static final String CLUSTER_STATUS_COMMAND
Config
String property: "cluster.statusCommand"protected static final String CLUSTER_MODULES
Config
List property: "cluster.modules"protected static final List<String> CLUSTER_NUM_PROCESSORS
Config
String property "cluster.jobHeader" to set number of cores.protected static final String CLUSTER_PROLOGUE
Config
String property: "cluster.prologue"protected static final String SCRIPT_JOB_HEADER
Config
String property: "cluster.jobHeader"public static void buildScripts(ScriptModule module, List<List<String>> data) throws PipelineScriptException
module
- ScriptModuledata
- Bash script linesPipelineScriptException
- if any errors occur writing module scriptprotected static List<String> buildExecuteFunction()
protected static void buildMainScript(ScriptModule module) throws ConfigException, IOException, DockerVolCreationException
module
- ScriptModuleIOException
- if errors occur writing the MAIN script linesConfigException
- if problems occur accessing Config
propertiesDockerVolCreationException
protected static List<String> buildScriptFailureFunction(String path)
path
- Script pathprotected static File createScript(ScriptModule module, String scriptPath, List<String> lines) throws IOException
module
- ScriptModulescriptPath
- Worker script pathlines
- Ordered bash script lines to write to the scriptPathIOException
- if unable to write the bash script to file systemprotected static String getMainScriptExecuteWorkerLine(String workerScriptPath)
workerScriptPath
- Worker script pathprotected static List<String> getWorkerScriptLines(List<String> lines)
lines
- Basic script lines generated by a BioModule.protected static String getWorkerScriptPath(ScriptModule module) throws ConfigNotFoundException, ConfigFormatException
module
- ScriptModule is the module that owns the scriptsConfigFormatException
- if "script.numWorkers" property is not a positive
integerConfigNotFoundException
- if "script.numWorkers" property is undefinedprotected static List<String> initMainScript(ScriptModule module) throws ConfigException, DockerVolCreationException
module
- ScriptModule script directory is where main script is writtenConfigException
- if errors occur accessing Config
propertiesDockerVolCreationException
protected static List<String> pathVariableVals(ScriptModule module)
insertPathVars
.module
- insertPathVars
protected static List<String> passEnvironmentVarVals(ScriptModule module) throws ConfigNotFoundException, ConfigFormatException
protected static List<String> insertPathVars(ScriptModule module, List<String> scriptLines)
pathVariableVals
.
Using these variables in place of full file paths makes the script easier to read.module
- scriptLines
- pathVariableVals
protected static List<String> initWorkerScript(ScriptModule module, String scriptPath) throws Exception
module
- ScriptModule script directory is where main script is writtenscriptPath
- Worker script absolute file pathException
- if required properties are undefinedprotected static void writeScript(ScriptModule module, BufferedWriter writer, List<String> scriptLines) throws IOException
module
- ScriptModulewriter
- BufferedWriter writes to the filescriptLines
- List or shell script linesIOException
- if errors occur writing the new scriptpublic static List<String> loadModules(ScriptModule module)
Config
."cluster.modules"module
- ScriptModulepublic static void registerProps() throws API_Exception
API_Exception
public static void checkDependencies(BioModule module) throws ConfigNotFoundException, ConfigFormatException