biolockj.module.implicit

Class ImportMetadata

java.lang.Object

biolockj.module.BioModuleImpl

biolockj.module.implicit.ImportMetadata

All Implemented Interfaces:

ApiModule, BioModule, Comparable<BioModule>

public class ImportMetadata
extends BioModuleImpl
implements ApiModule

This BioModule validates the contents/format of the project metadata file and the related Config properties. If successful, the metadata is saved as a tab delimited text file.

GUI Module Name

Import metadata

Field Summary

Fields inherited from class biolockj.module.BioModuleImpl

GZIP_EXT, LOG_EXT, PDF_EXT, RETURN, SH_EXT, TAB_DELIM, TSV_EXT, TXT_EXT

Fields inherited from interface biolockj.module.BioModule

LOG_DIR, MAIN_SCRIPT_PREFIX, NO_VERSION, OUTPUT_DIR, RES_DIR, TEMP_DIR

Constructor Summary

Constructors

Constructor and Description
ImportMetadata()

Method Summary

Modifier and Type	Method and Description
protected void	buildNewMetadataFile() Create a simple metadata file in the module output directory, with only the 1st column populated with Sample IDs.
void	checkDependencies() If restarting or running a direct pipeline execute the cleanup for completed modules.
void	cleanUp() Verify the metadata fields configured for R reports.
void	executeTask() If Config."metadata.filePath" is undefined, build a new metadata file with only 1 column of sample IDs.
protected String	formatMetaId(String sampleIdColumnName) Format the metadata ID to remove problematic invisible characters (particularly converted Excel files).
String	getCitationString() At a minimum, this should return the name and/or url for the wrapped tool.
String	getDescription() Briefly describe what this module does.
String	getDetails() A extension of getDescription.
String	getDockerImageName() Get the docker image to use for this module.
String	getDockerImageOwner() Get the name of the docker hub user that owns the image to use for this module.
protected String	getQuotedValue(String val) The member variable quotedText caches the input held within a quoted block.
protected TreeSet<String>	getSampleIds() Extract the sample IDs from the file names with SeqUtil.getSampleIdFromString(String)
String	getSummary() The metadata file can be updated several times during pipeline execution.
protected boolean	inQuotes(String val) Method called each time a line from metadata contains the Config."metadata.columnDelim".
protected String	parseRow(String line, boolean isHeader) Method called to parse a row from the metadata file, where Config."metadata.columnDelim" separates columns.
protected void	verifyAllRowsMapToSeqFile(List<File> files) Verify every row (every Sample ID) maps to a sequence file
protected static void	verifyHeader(String cell, List<String> colNames, int colNum) Verify column headers are not null and unique

Methods inherited from class biolockj.module.BioModuleImpl

addGeneralProperty, addGeneralProperty, addGeneralProperty, addNewProperty, addNewProperty, cacheInputFiles, compareTo, equals, findModuleInputFiles, getAlias, getDescription, getDockerImageTag, getFileCache, getID, getInputFiles, getLogDir, getMenuPlacement, getMetadata, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getPropDefault, getPropDescMap, getPropType, getPropTypeMap, getResourceDir, getTempDir, getTitle, hashCode, init, isValidInputModule, isValidProp, listProps, setAlias, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface biolockj.api.ApiModule

getDescription, getMenuPlacement, getPropType, getTitle, isValidProp, listProps

Methods inherited from interface biolockj.module.BioModule

getAlias, getDockerImageTag, getID, getInputFiles, getLogDir, getMetadata, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getPropDefault, getResourceDir, getTempDir, init, isValidInputModule, setAlias, version

Constructor Detail

ImportMetadata

public ImportMetadata()

Method Detail

checkDependencies

public void checkDependencies()
                       throws Exception

Description copied from class: BioModuleImpl

If restarting or running a direct pipeline execute the cleanup for completed modules.

Specified by:

checkDependencies in interface BioModule

Specified by:

checkDependencies in class BioModuleImpl

Throws:

Exception - thrown if missing or invalid dependencies are found

cleanUp

public void cleanUp()
             throws Exception

Verify the metadata fields configured for R reports.

Specified by:

cleanUp in interface BioModule

Overrides:

cleanUp in class BioModuleImpl

Throws:

Exception - thrown if any runtime error occurs

executeTask

public void executeTask()
                 throws Exception

If Config."metadata.filePath" is undefined, build a new metadata file with only 1 column of sample IDs. Otherwise, import "metadata.filePath" file and call MetaUtil.refreshCache() to validate, format, and cache metadata as a tab delimited text file.

Specified by:

executeTask in interface BioModule

Specified by:

executeTask in class BioModuleImpl

Throws:

Exception - thrown if the module is unable to complete is task

getSummary

public String getSummary()
                  throws Exception

The metadata file can be updated several times during pipeline execution. Summary prints the file-path of the final metadata file, along with sample and field count (obtained from MetaUtil).

Specified by:

getSummary in interface BioModule

Overrides:

getSummary in class BioModuleImpl

Returns:

Module summary of metadata path, #samples and #fields

Throws:

Exception - if any error occurs

buildNewMetadataFile

protected void buildNewMetadataFile()
                             throws BioLockJException

Create a simple metadata file in the module output directory, with only the 1st column populated with Sample IDs.

Throws:

BioLockJException

formatMetaId

protected String formatMetaId(String sampleIdColumnName)

Format the metadata ID to remove problematic invisible characters (particularly converted Excel files). If the first Excel cell value starts with # symbol, Excel adds a ZERO WIDTH NO-BREAK space as an invisible character. Here we strip this value; See http://www.fileformat.info/info/unicode/char/feff/index.htm

Parameters:

sampleIdColumnName - Current name of metadata Sample ID column

Returns:

Formatted Sample ID column name

getQuotedValue

protected String getQuotedValue(String val)

The member variable quotedText caches the input held within a quoted block. This block can contain the Config."metadata.columnDelim" which will be read as a character. If val closes an open quoted block, the entire quotedBlock is returned (ending with Config."metadata.columnDelim" as all cells do) and the quotedText cache is cleared.

Parameters:

val - Parameter to evaluate

Returns:

Quoted text block (so far)

getSampleIds

protected TreeSet<String> getSampleIds()
                                throws Exception

Extract the sample IDs from the file names with SeqUtil.getSampleIdFromString(String)

Returns:

Ordered set of Sample IDs

Throws:

Exception

inQuotes

protected boolean inQuotes(String val)

Method called each time a line from metadata contains the Config."metadata.columnDelim". If the Config."metadata.columnDelim" is encountered within a quoted block, it should be interpreted as a character (not interpreted as a column delimiter).

Parameters:

val - Parameter to evaluate

Returns:

true if within open quotes

parseRow

protected String parseRow(String line,
                          boolean isHeader)
                   throws Exception

Method called to parse a row from the metadata file, where Config."metadata.columnDelim" separates columns. The quotedText member variable serves as a cache to build cell values contained in quotes which may include the Config."metadata.columnDelim" as a standard character. Each row increments rowNum member variable. When the header row is processed, colNames caches the field names.

Parameters:

line - read from metadata file

isHeader - is true for only the first row

Returns:

parsed row value

Throws:

Exception - if required Config values are missing or invalid

verifyAllRowsMapToSeqFile

protected void verifyAllRowsMapToSeqFile(List<File> files)
                                  throws Exception

Verify every row (every Sample ID) maps to a sequence file

Parameters:

files - List of sequence files

Throws:

ConfigViolationException - if unmapped Sample IDs are found

Exception - if other errors occur

verifyHeader

protected static void verifyHeader(String cell,
                                   List<String> colNames,
                                   int colNum)
                            throws Exception

Verify column headers are not null and unique

Parameters:

cell - value of the header column name

colNames - a list of column names read so far

colNum - included for reference in error message if needed

Throws:

Exception - if a column header is null or not unique

getDockerImageOwner

public String getDockerImageOwner()

Description copied from interface: BioModule

Get the name of the docker hub user that owns the image to use for this module. Docker images from docker hub are specified using this syntax: /:

Specified by:

getDockerImageOwner in interface BioModule

Overrides:

getDockerImageOwner in class BioModuleImpl

Returns:

getDockerImageName

public String getDockerImageName()

Description copied from interface: BioModule

Get the docker image to use for this module. The owner and version/tag are specificed separately; this is just the image name. Docker images from docker hub are specified using this syntax: /:

Specified by:

getDockerImageName in interface BioModule

Returns:

getDescription

public String getDescription()

Description copied from interface: ApiModule

Briefly describe what this module does. A more detailed description may be returned by getDetails.

Specified by:

getDescription in interface ApiModule

Returns:

getCitationString

public String getCitationString()

Description copied from interface: ApiModule

At a minimum, this should return the name and/or url for the wrapped tool. For BioLockJ home-grown modules, it can cite BioLockJ and give the current version. Ideally this will include the version for the wrapped tool. Tool version may require using a stored variable that can be filled in during execution; and the string that is returned pre-execution may not have the version.

Specified by:

getCitationString in interface ApiModule

Returns:

getDetails

public String getDetails()

Description copied from interface: ApiModule

A extension of getDescription. Beyond the brief description, give details such as the interaction between properties.

Specified by:

getDetails in interface ApiModule

Overrides:

getDetails in class BioModuleImpl

Returns: