Package weka.core.converters

# Class TextDirectoryLoader

java.lang.Object
weka.core.converters.AbstractLoader
weka.core.converters.TextDirectoryLoader

All Implemented Interfaces:
  `Serializable`, `CommandlineRunnable`, `BatchConverter`, `IncrementalConverter`, `Loader`, `OptionHandler`, `RevisionHandler`

public class TextDirectoryLoader extends AbstractLoader implements BatchConverter, IncrementalConverter, OptionHandler, CommandlineRunnable
Loads all text files in a directory and uses the subdirectory names as class labels. The content of the text files will be stored in a String attribute, the filename can be stored as well.

Valid options are:

     -D
      Enables debug output.
      (default: off)

     -F
      Stores the filename in an additional attribute.
      (default: off)

     -dir <directory>
      The directory to work on.
      (default: current directory)

     -charset <charset name>
      The character set to use, e.g UTF-8.
      (default: use the default character set)

     -R
      Retain all string attribute values when reading incrementally.

Based on code from the TextDirectoryToArff tool. See the
- Wiki article

Version:
  $Revision: 15255 $
Author:
  Ashraf M. Kibriya (amk14 at cs.waikato.ac.nz), Richard Kirkby (rkirkby at cs.waikato.ac.nz), fracpete (fracpete at waikato dot ac dot nz)
See Also:
  - `Loader`
  - Serialized Form

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface weka.core.converters.Loader

  `Loader.StructureNotReadyException`

- ## Field Summary

  ### Fields inherited from interface weka.core.converters.Loader

  `BATCH, INCREMENTAL, NONE`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `TextDirectoryLoader()`
  default constructor

- ## Method Summary

  Modifier and Type
  Method
  Description
  `String`
  `charSetTipText()`
  the tip text for this property
  `String`
  `debugTipText()`
  the tip text for this property
  `String`
  `getCharSet()`
  Get the character set to use when reading text files.
  `Instances`
  `getDataSet()`
  Return the full data set.
  `boolean`
  `getDebug()`
  Gets whether additional debug information is printed.
  `File`
  `getDirectory()`
  get the Dir specified as the source
  `String`
  `getFileDescription()`
  Returns a description of the file type, actually it's directories.
  `Instance`
  `getNextInstance(Instances structure)`
  Process input directories/files incrementally.
  `String[]`
  `getOptions()`
  Gets the setting
  `boolean`
  `getOutputFilename()`
  Gets whether the filename will be stored as an extra attribute.
  `String`
  `getRevision()`
  Returns the revision string.
  `Instances`
  `getStructure()`
  Determines and returns (if possible) the structure (internally the header) of the data set as an empty set of instances.
  `String`
  `globalInfo()`
  Returns a string describing this loader
  `Enumeration<Option>`
  `listOptions()`
  Lists the available options
  `static void`
  `main(String[] args)`
  Main method.
  `String`
  `outputFilenameTipText()`
  the tip text for this property
  `void`
  `postExecution()`
  Perform any teardown stuff that might need to happen after execution.
  `void`
  `preExecution()`
  Perform any setup stuff that might need to happen before commandline execution.
  `void`
  `reset()`
  Resets the loader ready to read a new data set
  `void`
  `run(Object toRun, String[] args)`
  Execute the supplied object.
  `void`
  `setCharSet(String charSet)`
  Set the character set to use when reading text files (an empty string indicates that the default character set will be used).
  `void`
  `setDebug(boolean value)`
  Sets whether to print some debug information.
  `void`
  `setDirectory(File dir)`
  sets the source directory
  `void`
  `setOptions(String[] options)`
  Parses a given list of options.
  `void`
  `setOutputFilename(boolean value)`
  Sets whether the filename will be stored as an extra attribute.
  `void`
  `setSource(File dir)`
  Resets the Loader object and sets the source of the data set to be the supplied File object.

  ### Methods inherited from class weka.core.converters.AbstractLoader

  `setRetrieval, setSource`

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### TextDirectoryLoader

    public TextDirectoryLoader()
    default constructor

- ## Method Details

  - ### globalInfo

    public String globalInfo()
    Returns a string describing this loader
    Returns:
      a description of the evaluator suitable for displaying in the explorer/experimenter gui

  - ### listOptions

    public Enumeration<Option> listOptions()
    Lists the available options
    Specified by:
      `listOptions` in interface `OptionHandler`
    Returns:
      an enumeration of the available options

  - ### setOptions

    public void setOptions(String[] options) throws Exception
    Parses a given list of options.

    Valid options are:

         -D
          Enables debug output.
          (default: off)

         -F
          Stores the filename in an additional attribute.
          (default: off)

         -dir <directory>
          The directory to work on.
          (default: current directory)

         -charset <charset name>
          The character set to use, e.g UTF-8.
          (default: use the default character set)

    Specified by:
      `setOptions` in interface `OptionHandler`
    Parameters:
      `options` - the options
    Throws:
      `Exception` - if options cannot be set

  - ### getOptions

    public String[] getOptions()
    Gets the setting
    Specified by:
      `getOptions` in interface `OptionHandler`
    Returns:
      the current setting

  - ### charSetTipText

    public String charSetTipText()
    the tip text for this property
    Returns:
      the tip text

  - ### setCharSet

    public void setCharSet(String charSet)
    Set the character set to use when reading text files (an empty string indicates that the default character set will be used).
    Parameters:
      `charSet` - the character set to use.

  - ### getCharSet

    public String getCharSet()
    Get the character set to use when reading text files. An empty string indicates that the default character set will be used.
    Returns:
      the character set name to use (or empty string to indicate that the default character set will be used).

  - ### setDebug

    public void setDebug(boolean value)
    Sets whether to print some debug information.
    Parameters:
      `value` - if true additional debug information will be printed.

  - ### getDebug

    public boolean getDebug()
    Gets whether additional debug information is printed.
    Returns:
      true if additional debug information is printed

  - ### debugTipText

    public String debugTipText()
    the tip text for this property
    Returns:
      the tip text

  - ### setOutputFilename

    public void setOutputFilename(boolean value)
    Sets whether the filename will be stored as an extra attribute.
    Parameters:
      `value` - if true the filename will be stored in an extra attribute

  - ### getOutputFilename

    public boolean getOutputFilename()
    Gets whether the filename will be stored as an extra attribute.
    Returns:
      true if the filename is stored in an extra attribute

  - ### outputFilenameTipText

    public String outputFilenameTipText()
    the tip text for this property
    Returns:
      the tip text

  - ### getFileDescription

    public String getFileDescription()
    Returns a description of the file type, actually it's directories.
    Returns:
      a short file description

  - ### getDirectory

    public File getDirectory()
    get the Dir specified as the source
    Returns:
      the source directory

  - ### setDirectory

    public void setDirectory(File dir) throws IOException
    sets the source directory
    Parameters:
      `dir` - the source directory
    Throws:
      `IOException` - if an error occurs

  - ### reset

    public void reset()
    Resets the loader ready to read a new data set
    Specified by:
      `reset` in interface `Loader`
    Overrides:
      `reset` in class `AbstractLoader`

  - ### setSource

    public void setSource(File dir) throws IOException
    Resets the Loader object and sets the source of the data set to be the supplied File object.
    Specified by:
      `setSource` in interface `Loader`
    Overrides:
      `setSource` in class `AbstractLoader`
    Parameters:
      `dir` - the source directory.
    Throws:
      `IOException` - if an error occurs

  - ### getStructure

    public Instances getStructure() throws IOException
    Determines and returns (if possible) the structure (internally the header) of the data set as an empty set of instances.
    Specified by:
      `getStructure` in interface `Loader`
    Specified by:
      `getStructure` in class `AbstractLoader`
    Returns:
      the structure of the data set as an empty set of Instances
    Throws:
      `IOException` - if an error occurs

  - ### getDataSet

    public Instances getDataSet() throws IOException
    Return the full data set. If the structure hasn't yet been determined by a call to getStructure then method should do so before processing the rest of the data set.
    Specified by:
      `getDataSet` in interface `Loader`
    Specified by:
      `getDataSet` in class `AbstractLoader`
    Returns:
      the structure of the data set as an empty set of Instances
    Throws:
      `IOException` - if there is no source or parsing fails

  - ### getNextInstance

    public Instance getNextInstance(Instances structure) throws IOException
    Process input directories/files incrementally.
    Specified by:
      `getNextInstance` in interface `Loader`
    Specified by:
      `getNextInstance` in class `AbstractLoader`
    Parameters:
      `structure` - ignored
    Returns:
      never returns without throwing an exception
    Throws:
      `IOException` - if a problem occurs

  - ### getRevision

    public String getRevision()
    Returns the revision string.
    Specified by:
      `getRevision` in interface `RevisionHandler`
    Returns:
      the revision

  - ### main

    public static void main(String[] args)
    Main method.
    Parameters:
      `args` - should contain the name of an input file.

  - ### preExecution

    public void preExecution() throws Exception
    Perform any setup stuff that might need to happen before commandline execution. Subclasses should override if they need to do something here
    Specified by:
      `preExecution` in interface `CommandlineRunnable`
    Throws:
      `Exception` - if a problem occurs during setup

  - ### postExecution

    public void postExecution() throws Exception
    Perform any teardown stuff that might need to happen after execution. Subclasses should override if they need to do something here
    Specified by:
      `postExecution` in interface `CommandlineRunnable`
    Throws:
      `Exception` - if a problem occurs during teardown

  - ### run

    public void run(Object toRun, String[] args) throws IllegalArgumentException
    Description copied from interface: `CommandlineRunnable`
    Execute the supplied object.
    Specified by:
      `run` in interface `CommandlineRunnable`
    Parameters:
      `toRun` - the object to execute
      `args` - any options to pass to the object
    Throws:
      `IllegalArgumentException`