public class NextflowUtil extends Object
Modifier and Type | Field and Description |
---|---|
static String |
AWS_CONFIG_DIR
Name of the AWS S3 sub-directory used to save pipeline reports
|
static String |
AWS_COPY_DB_TO_S3
Config Boolean property:"aws.copyDbToS3" |
static String |
AWS_COPY_PIPELINE_TO_S3
Config Boolean property: "aws.copyPipelineToS3" |
static String |
AWS_COPY_REPORTS_TO_S3
|
static String |
AWS_PURGE_EFS_INPUTS
Config Boolean property: "aws.purgeEfsInputs" |
static String |
AWS_PURGE_EFS_OUTPUT
Config Boolean property: "aws.purgeEfsOutput" |
static String |
AWS_RAM
Config String property: "aws.ram" |
static String |
AWS_REPORT_DIR
Name of the AWS S3 subdirectory used to save pipeline reports
|
static String |
AWS_S3
Config String property: "aws.s3" |
protected static String |
EC2_ACQUISITION_STRATEGY
Config String property: "aws.ec2AcquisitionStrategy" |
protected static String |
EC2_INSTANCE_TYPE
Config String property: "aws.ec2InstanceType" |
protected static String |
NF_LOG
The Docker container will generate a nextflow.log file in the root directory, this is the file name
|
Constructor and Description |
---|
NextflowUtil() |
Modifier and Type | Method and Description |
---|---|
static void |
awsSyncS3(String efsPath,
boolean waitUntilComplete)
Sync file or directory with S3 bucket.
Dir example: aws s3 sync $EFS/config s3://blj-2019-04-05/config File example: aws s3 cp ${BLJ_META}/testMetadata.tsv s3://blj-2019-04-05/metadata/testMetadata.tsv Register each efsPath to avoid syncing the same path more than once. |
static File |
getMainNf()
Get the Nextflow main.nf file path.
|
protected static List<String> |
getNextflowLines(File template)
Build the main.nf lines from the template file by replacing several parameters.
|
static File |
getNfReportDir()
Get the Nextflow report directory
|
static int |
getS3_TransferTimeout()
Get S3 Transfer timeout limit - default = 30 minutes if undefined
|
static boolean |
nextflowLogExists()
Return true if the Nextflow log has been saved to the pipeline root directory
|
static boolean |
purgeEfsData()
Purge EFS data based on pipeline Config.
|
static void |
registerProps()
Register properties with the Properties class for API access.
|
static boolean |
saveEfsDataToS3()
Save EFS data to S3 based on pipeline Config.
|
static void |
saveNextflowLog()
Save a copy of the Nextflow log file to the Pipeline root directory
|
static void |
saveNextflowSuccessFlag()
Save success flag, so after pipeline bash start script can stop/terminate S3 instances if successful.
|
static void |
stageRootConfig()
Before any AWS or Nextflow functionality can be used, the Docker root user $HOME directory must be updated with
the EC2 user aws + Nextflow config.
|
static void |
startNextflow(List<BioModule> modules)
Call this method to build the Nextflow main.nf for the current pipeline.
|
static void |
stopNextflow()
Stop Nextflow process (required since parent Java process that ran BioLockJ pipeline will not halt until this
subprocess is finished.
|
public static final String AWS_CONFIG_DIR
public static final String AWS_COPY_DB_TO_S3
Config
Boolean property:"aws.copyDbToS3"public static final String AWS_COPY_PIPELINE_TO_S3
Config
Boolean property: "aws.copyPipelineToS3"public static final String AWS_COPY_REPORTS_TO_S3
public static final String AWS_PURGE_EFS_INPUTS
Config
Boolean property: "aws.purgeEfsInputs"public static final String AWS_PURGE_EFS_OUTPUT
Config
Boolean property: "aws.purgeEfsOutput"public static final String AWS_RAM
Config
String property: "aws.ram"public static final String AWS_REPORT_DIR
public static final String AWS_S3
Config
String property: "aws.s3"protected static final String EC2_ACQUISITION_STRATEGY
Config
String property: "aws.ec2AcquisitionStrategy"protected static final String EC2_INSTANCE_TYPE
Config
String property: "aws.ec2InstanceType"protected static final String NF_LOG
public static void awsSyncS3(String efsPath, boolean waitUntilComplete) throws Exception
efsPath
- File or directory to syncwaitUntilComplete
- Boolean if enabled will block until process completes before moving onException
- if errors occurpublic static File getMainNf()
public static File getNfReportDir()
public static int getS3_TransferTimeout()
public static boolean nextflowLogExists()
public static boolean purgeEfsData()
public static boolean saveEfsDataToS3()
public static void saveNextflowLog()
public static void saveNextflowSuccessFlag()
public static void stageRootConfig() throws IOException
IOException
- if source or target config directories are not foundpublic static void startNextflow(List<BioModule> modules) throws Exception
modules
- Pipeline modulesException
- if errors occurpublic static void stopNextflow()
protected static List<String> getNextflowLines(File template) throws Exception
template
- Generated .main.nf templateException
- if errors occurpublic static void registerProps() throws API_Exception
API_Exception