biolockj.module.seq

Class SeqFileValidator

java.lang.Object

biolockj.module.BioModuleImpl

biolockj.module.ScriptModuleImpl

biolockj.module.JavaModuleImpl

biolockj.module.seq.SeqFileValidator

All Implemented Interfaces:

ApiModule, BioModule, JavaModule, ReadCounter, ScriptModule, SeqModule, Comparable<BioModule>

public class SeqFileValidator
extends JavaModuleImpl
implements SeqModule, ApiModule, ReadCounter

This BioModule validates fasta/fastq file formats are valid and enforces min/max read lengths.

GUI Module Name

Sequence File Validator

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	INPUT_SEQ_MAX Config Integer property "seqFileValidator.seqMaxLen" defines the maximum number of bases per read
protected static String	INPUT_SEQ_MIN Config Integer property "seqFileValidator.seqMinLen" defines the minimum number of bases per read
static String	NUM_VALID_READS Column name that holds number of valid reads per sample: "Num_Valid_Reads"
protected static String	REQUIRE_EUQL_NUM_PAIRS Config Boolean property "seqFileValidator.requireEqualNumPairs" determines if module requires equal number of forward and reverse reads (simple check).

Fields inherited from class biolockj.module.JavaModuleImpl

BLJ_OPTIONS

Fields inherited from class biolockj.module.BioModuleImpl

GZIP_EXT, LOG_EXT, PDF_EXT, RETURN, SH_EXT, TAB_DELIM, TSV_EXT, TXT_EXT

Fields inherited from interface biolockj.module.BioModule

LOG_DIR, MAIN_SCRIPT_PREFIX, NO_VERSION, OUTPUT_DIR, RES_DIR, TEMP_DIR

Constructor Summary

Constructors

Constructor and Description
SeqFileValidator()

Method Summary

Modifier and Type	Method and Description
void	checkDependencies() Validate module dependencies: Require Config. exists Require Config. is positive integer Require Config. is positive integer Verify Config. is positive integer if set Start the AWS DB sync to S3 if a novel DB has been configure and "aws.copyDbToS3" is enabled
void	cleanUp() Set "Num_Valid_Reads" as the number of reads field.
String	getCitationString() At a minimum, this should return the name and/or url for the wrapped tool.
String	getDescription() Briefly describe what this module does.
List<File>	getSeqFiles(Collection<File> files) Return only sequence files for sample IDs found in the metadata file. If Config."metadata.required" = "Y", an error is thrown to list the files that cannot be matched to a metadata row.
String	getSummary() Produce a summary message with counts on total number of reads and number of valid reads containing a barcode defined in the metadata file.
Boolean	isValidProp(String property) Tests to see if the value val is valid for property prop; primarily tests format.
protected void	removeBadFiles() Remove sequence files in which all reads failed validation checks, leaving only an empty file.
void	runModule() Cache sampleIds to compare to validated sampleIds post-processing.
protected void	validateFile(File file, Integer fileCount) Validate sequence files: Validate valid 1st sequence header character is expected character Validate fastq files have same number of bases and quality scores per read Remove reads below minimum threshold: "seqFileValidator.seqMinLen" Trim reads if above the maximum threshold: "seqFileValidator.seqMaxLen" Invalid reads are saved to a file in the module temp directory for analysis/review.
protected void	verifyPairedSeqs() Verify equal number of forward and reverse read files. if "seqFileValidator.requireEqualNumPairs"="Y", verify forward and reverse read files have an equal number of reads.

Methods inherited from class biolockj.module.JavaModuleImpl

buildScript, executeTask, getDockerImageName, getDockerImageOwner, getDockerImageTag, getWorkerScriptFunctions, isValidInputModule, markStatus, moduleComplete, moduleFailed, runBioLockJ_CMD

Methods inherited from class biolockj.module.ScriptModuleImpl

buildScriptForPairedReads, getJobParams, getMainScript, getRuntimeParams, getScriptDir, getScriptErrors, getTimeout, hasScripts

Methods inherited from class biolockj.module.BioModuleImpl

addGeneralProperty, addGeneralProperty, addGeneralProperty, addNewProperty, addNewProperty, cacheInputFiles, compareTo, equals, findModuleInputFiles, getAlias, getDescription, getDetails, getFileCache, getID, getInputFiles, getLogDir, getMenuPlacement, getMetadata, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getPropDefault, getPropDescMap, getPropType, getPropTypeMap, getResourceDir, getTempDir, getTitle, hashCode, init, listProps, setAlias, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface biolockj.module.ScriptModule

buildScript, buildScriptForPairedReads, getJobParams, getMainScript, getScriptDir, getScriptErrors, getTimeout, getWorkerScriptFunctions

Methods inherited from interface biolockj.module.BioModule

executeTask, getAlias, getDockerImageName, getDockerImageOwner, getDockerImageTag, getID, getInputFiles, getLogDir, getMetadata, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getPropDefault, getResourceDir, getTempDir, init, isValidInputModule, setAlias, version

Methods inherited from interface biolockj.api.ApiModule

getDescription, getDetails, getMenuPlacement, getPropType, getTitle, listProps

Field Detail

NUM_VALID_READS

public static final String NUM_VALID_READS

Column name that holds number of valid reads per sample: "Num_Valid_Reads"

See Also:

Constant Field Values

INPUT_SEQ_MAX

protected static final String INPUT_SEQ_MAX

Config Integer property "seqFileValidator.seqMaxLen" defines the maximum number of bases per read

See Also:

Constant Field Values

INPUT_SEQ_MIN

protected static final String INPUT_SEQ_MIN

Config Integer property "seqFileValidator.seqMinLen" defines the minimum number of bases per read

See Also:

Constant Field Values

REQUIRE_EUQL_NUM_PAIRS

protected static final String REQUIRE_EUQL_NUM_PAIRS

Config Boolean property "seqFileValidator.requireEqualNumPairs" determines if module requires equal number of forward and reverse reads (simple check).

See Also:

Constant Field Values

Constructor Detail

SeqFileValidator

public SeqFileValidator()

Method Detail

checkDependencies

public void checkDependencies()
                       throws Exception

Description copied from class: ScriptModuleImpl

Validate module dependencies:

• Require Config. exists
• Require Config. is positive integer
• Require Config. is positive integer
• Verify Config. is positive integer if set
• Start the AWS DB sync to S3 if a novel DB has been configure and "aws.copyDbToS3" is enabled

Specified by:

checkDependencies in interface BioModule

Overrides:

checkDependencies in class ScriptModuleImpl

Throws:

Exception - thrown if missing or invalid dependencies are found

isValidProp

public Boolean isValidProp(String property)
                    throws Exception

Description copied from interface: ApiModule

Tests to see if the value val is valid for property prop; primarily tests format. This is designed for individual properties. To make sure property values are compatible, use BioModule.checkDependencies(). Using switch/case or a stack of if/else is recommended. Within each case, call any/all method that is used by this module to access the value from the config file, leveraging the checks in the Config.get* methods. This method should never actually return false. If the value is not valid, it should throw an exception that includes a helpful message about whats not valid. As part of a throwable, that message is passed along to wherever the call started. Any time that "false" is actually the desired form, this method should be wrapped in a try/catch.

Specified by:

isValidProp in interface ApiModule

Overrides:

isValidProp in class ScriptModuleImpl

Returns:

true if value is recognized and good, false/Exception if it is recognized and invalid, null if prop is not recognized.

Throws:

Exception

cleanUp

public void cleanUp()
             throws Exception

Set "Num_Valid_Reads" as the number of reads field.

Specified by:

cleanUp in interface BioModule

Overrides:

cleanUp in class BioModuleImpl

Throws:

Exception - thrown if any runtime error occurs

getSeqFiles

public List<File> getSeqFiles(Collection<File> files)
                       throws SequnceFormatException

Description copied from interface: SeqModule

Return only sequence files for sample IDs found in the metadata file.
If Config."metadata.required" = "Y", an error is thrown to list the files that cannot be matched to a metadata row.

Specified by:

getSeqFiles in interface SeqModule

Parameters:

files - Module input files

Returns:

Module sequence files

Throws:

SequnceFormatException - If Config."metadata.required" = "Y" but sequence files found that do not have a corresponding record in the metadata file or if invalid metadata prevents parsing SEQ files.

getSummary

public String getSummary()
                  throws Exception

Produce a summary message with counts on total number of reads and number of valid reads containing a barcode defined in the metadata file.

Specified by:

getSummary in interface BioModule

Overrides:

getSummary in class ScriptModuleImpl

Returns:

Summary of BioModule execution

Throws:

Exception - if any error occurs

runModule

public void runModule()
               throws Exception

Cache sampleIds to compare to validated sampleIds post-processing. Call validateFile(File, Integer) for each input file.
Call removeBadFiles() to remove empty files (cases where all reads fail validation).
Call verifyPairedSeqs() if module input files are paired read files.
Call MetaUtil.addColumn(String, Map, File, boolean)

Specified by:

runModule in interface JavaModule

Specified by:

runModule in class JavaModuleImpl

Throws:

Exception - thrown if any runtime error occurs

removeBadFiles

protected void removeBadFiles()

Remove sequence files in which all reads failed validation checks, leaving only an empty file.

validateFile

protected void validateFile(File file,
                            Integer fileCount)
                     throws Exception

Validate sequence files:

1. Validate valid 1st sequence header character is expected character
2. Validate fastq files have same number of bases and quality scores per read
3. Remove reads below minimum threshold: "seqFileValidator.seqMinLen"
4. Trim reads if above the maximum threshold: "seqFileValidator.seqMaxLen"

Invalid reads are saved to a file in the module temp directory for analysis/review.

Parameters:

file - Sequence file

fileCount - Integer count

Throws:

Exception - if I/O errors occur while processing sequence files

verifyPairedSeqs

protected void verifyPairedSeqs()
                         throws Exception

Verify equal number of forward and reverse read files.
if "seqFileValidator.requireEqualNumPairs"="Y", verify forward and reverse read files have an equal number of reads.

Throws:

Exception - if validations fail or errors occur

getDescription

public String getDescription()

Description copied from interface: ApiModule

Briefly describe what this module does. A more detailed description may be returned by getDetails.

Specified by:

getDescription in interface ApiModule

Returns:

getCitationString

public String getCitationString()

Description copied from interface: ApiModule

At a minimum, this should return the name and/or url for the wrapped tool. For BioLockJ home-grown modules, it can cite BioLockJ and give the current version. Ideally this will include the version for the wrapped tool. Tool version may require using a stored variable that can be filled in during execution; and the string that is returned pre-execution may not have the version.

Specified by:

getCitationString in interface ApiModule

Returns: