NextflowUtil

java.lang.Object
- biolockj.util.NextflowUtil

```
public class NextflowUtil
extends Object
```
This utility builds the Nextflow main.nf used in AWS pipelines.

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`AWS_CONFIG_DIR` Name of the AWS S3 sub-directory used to save pipeline reports
`static String`	`AWS_COPY_DB_TO_S3` `Config` Boolean property:"aws.copyDbToS3"
`static String`	`AWS_COPY_PIPELINE_TO_S3` `Config` Boolean property: "aws.copyPipelineToS3"
`static String`	`AWS_COPY_REPORTS_TO_S3` `Config` Boolean property: "aws.copyPipelineToS3"
`static String`	`AWS_PURGE_EFS_INPUTS` `Config` Boolean property: "aws.purgeEfsInputs"
`static String`	`AWS_PURGE_EFS_OUTPUT` `Config` Boolean property: "aws.purgeEfsOutput"
`static String`	`AWS_RAM` `Config` String property: "aws.ram"
`static String`	`AWS_REPORT_DIR` Name of the AWS S3 subdirectory used to save pipeline reports
`static String`	`AWS_S3` `Config` String property: "aws.s3"
`protected static String`	`EC2_ACQUISITION_STRATEGY` `Config` String property: "aws.ec2AcquisitionStrategy"
`protected static String`	`EC2_INSTANCE_TYPE` `Config` String property: "aws.ec2InstanceType"
`protected static String`	`NF_LOG` The Docker container will generate a nextflow.log file in the root directory, this is the file name

Constructor Summary

Constructors
Constructor and Description

NextflowUtil()

Constructors
Constructor and Description
`NextflowUtil()`

Method Summary

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static void`	`awsSyncS3(String efsPath, boolean waitUntilComplete)` Sync file or directory with S3 bucket. Dir example: aws s3 sync $EFS/config s3://blj-2019-04-05/config File example: aws s3 cp ${BLJ_META}/testMetadata.tsv s3://blj-2019-04-05/metadata/testMetadata.tsv Register each efsPath to avoid syncing the same path more than once.
`static File`	`getMainNf()` Get the Nextflow main.nf file path.
`protected static List<String>`	`getNextflowLines(File template)` Build the main.nf lines from the template file by replacing several parameters.
`static File`	`getNfReportDir()` Get the Nextflow report directory
`static int`	`getS3_TransferTimeout()` Get S3 Transfer timeout limit - default = 30 minutes if undefined
`static boolean`	`nextflowLogExists()` Return true if the Nextflow log has been saved to the pipeline root directory
`static boolean`	`purgeEfsData()` Purge EFS data based on pipeline Config.
`static void`	`registerProps()` Register properties with the Properties class for API access.
`static boolean`	`saveEfsDataToS3()` Save EFS data to S3 based on pipeline Config.
`static void`	`saveNextflowLog()` Save a copy of the Nextflow log file to the Pipeline root directory
`static void`	`saveNextflowSuccessFlag()` Save success flag, so after pipeline bash start script can stop/terminate S3 instances if successful.
`static void`	`stageRootConfig()` Before any AWS or Nextflow functionality can be used, the Docker root user $HOME directory must be updated with the EC2 user aws + Nextflow config.
`static void`	`startNextflow(List<BioModule> modules)` Call this method to build the Nextflow main.nf for the current pipeline.
`static void`	`stopNextflow()` Stop Nextflow process (required since parent Java process that ran BioLockJ pipeline will not halt until this subprocess is finished.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - AWS_CONFIG_DIR
```
public static final String AWS_CONFIG_DIR
```
    Name of the AWS S3 sub-directory used to save pipeline reports
    
    See Also:
    
    Constant Field Values
  - AWS_COPY_DB_TO_S3
```
public static final String AWS_COPY_DB_TO_S3
```
    Config Boolean property:"aws.copyDbToS3"
    
    See Also:
    
    Constant Field Values
  - AWS_COPY_PIPELINE_TO_S3
```
public static final String AWS_COPY_PIPELINE_TO_S3
```
    Config Boolean property: "aws.copyPipelineToS3"
    
    See Also:
    
    Constant Field Values
  - AWS_COPY_REPORTS_TO_S3
```
public static final String AWS_COPY_REPORTS_TO_S3
```
    Config Boolean property: "aws.copyPipelineToS3"
    
    See Also:
    
    Constant Field Values
  - AWS_PURGE_EFS_INPUTS
```
public static final String AWS_PURGE_EFS_INPUTS
```
    Config Boolean property: "aws.purgeEfsInputs"
    
    See Also:
    
    Constant Field Values
  - AWS_PURGE_EFS_OUTPUT
```
public static final String AWS_PURGE_EFS_OUTPUT
```
    Config Boolean property: "aws.purgeEfsOutput"
    
    See Also:
    
    Constant Field Values
  - AWS_RAM
```
public static final String AWS_RAM
```
    Config String property: "aws.ram"
    
    See Also:
    
    Constant Field Values
  - AWS_REPORT_DIR
```
public static final String AWS_REPORT_DIR
```
    Name of the AWS S3 subdirectory used to save pipeline reports
    
    See Also:
    
    Constant Field Values
  - AWS_S3
```
public static final String AWS_S3
```
    Config String property: "aws.s3"
    
    See Also:
    
    Constant Field Values
  - EC2_ACQUISITION_STRATEGY
```
protected static final String EC2_ACQUISITION_STRATEGY
```
    Config String property: "aws.ec2AcquisitionStrategy"
    
    See Also:
    
    Constant Field Values
  - EC2_INSTANCE_TYPE
```
protected static final String EC2_INSTANCE_TYPE
```
    Config String property: "aws.ec2InstanceType"
    
    See Also:
    
    Constant Field Values
  - NF_LOG
```
protected static final String NF_LOG
```
    The Docker container will generate a nextflow.log file in the root directory, this is the file name
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - NextflowUtil
```
public NextflowUtil()
```
- Method Detail
  - awsSyncS3
```
public static void awsSyncS3(String efsPath,
                             boolean waitUntilComplete)
                      throws Exception
```
    Sync file or directory with S3 bucket.
    Dir example: aws s3 sync $EFS/config s3://blj-2019-04-05/config
    File example: aws s3 cp ${BLJ_META}/testMetadata.tsv s3://blj-2019-04-05/metadata/testMetadata.tsv Register each efsPath to avoid syncing the same path more than once.
    
    Parameters:
    
    efsPath - File or directory to sync
    
    waitUntilComplete - Boolean if enabled will block until process completes before moving on
    
    Throws:
    
    Exception - if errors occur
  - getMainNf
```
public static File getMainNf()
```
    Get the Nextflow main.nf file path.
    
    Returns:
    
    Nextflow main.nf
  - getNfReportDir
```
public static File getNfReportDir()
```
    Get the Nextflow report directory
    
    Returns:
    
    Nextflow report directory
  - getS3_TransferTimeout
```
public static int getS3_TransferTimeout()
```
    Get S3 Transfer timeout limit - default = 30 minutes if undefined
    
    Returns:
    
    Number of minutes beforer S3 transfer thread will abort
  - nextflowLogExists
```
public static boolean nextflowLogExists()
```
    Return true if the Nextflow log has been saved to the pipeline root directory
    
    Returns:
    
    TRUE if log exists in pipeline root directory
  - purgeEfsData
```
public static boolean purgeEfsData()
```
    Purge EFS data based on pipeline Config.
    
    Returns:
    
    TRUE if not errors occur
  - saveEfsDataToS3
```
public static boolean saveEfsDataToS3()
```
    Save EFS data to S3 based on pipeline Config.
    
    Returns:
    
    TRUE if not errors occur
  - saveNextflowLog
```
public static void saveNextflowLog()
```
    Save a copy of the Nextflow log file to the Pipeline root directory
  - saveNextflowSuccessFlag
```
public static void saveNextflowSuccessFlag()
```
    Save success flag, so after pipeline bash start script can stop/terminate S3 instances if successful.
  - stageRootConfig
```
public static void stageRootConfig()
                            throws IOException
```
    Before any AWS or Nextflow functionality can be used, the Docker root user $HOME directory must be updated with the EC2 user aws + Nextflow config.
    
    Throws:
    
    IOException - if source or target config directories are not found
  - startNextflow
```
public static void startNextflow(List<BioModule> modules)
                          throws Exception
```
    Call this method to build the Nextflow main.nf for the current pipeline.
    
    Parameters:
    
    modules - Pipeline modules
    
    Throws:
    
    Exception - if errors occur
  - stopNextflow
```
public static void stopNextflow()
```
    Stop Nextflow process (required since parent Java process that ran BioLockJ pipeline will not halt until this subprocess is finished. Also create pipeline success flag file in $HOME dir on the EC2 head node.
  - getNextflowLines
```
protected static List<String> getNextflowLines(File template)
                                        throws Exception
```
    Build the main.nf lines from the template file by replacing several parameters.
    
    Parameters:
    
    template - Generated .main.nf template
    
    Returns:
    
    List of lines to save in final main.nf
    
    Throws:
    
    Exception - if errors occur
  - registerProps
```
public static void registerProps()
                          throws API_Exception
```
    Register properties with the Properties class for API access.
    
    Throws:
    
    API_Exception

Class NextflowUtil

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

AWS_CONFIG_DIR

AWS_COPY_DB_TO_S3

AWS_COPY_PIPELINE_TO_S3

AWS_COPY_REPORTS_TO_S3

AWS_PURGE_EFS_INPUTS

AWS_PURGE_EFS_OUTPUT

AWS_RAM

AWS_REPORT_DIR

AWS_S3

EC2_ACQUISITION_STRATEGY

EC2_INSTANCE_TYPE

NF_LOG

Constructor Detail

NextflowUtil

Method Detail

awsSyncS3

getMainNf

getNfReportDir

getS3_TransferTimeout

nextflowLogExists

purgeEfsData

saveEfsDataToS3

saveNextflowLog

saveNextflowSuccessFlag

stageRootConfig

startNextflow

stopNextflow

getNextflowLines

registerProps