Package weka.knowledgeflow.steps

# Class MakeResourceIntensive

java.lang.Object
weka.knowledgeflow.steps.BaseStep
weka.knowledgeflow.steps.MakeResourceIntensive

All Implemented Interfaces:
  `Serializable`, `BaseStepExtender`, `Step`

@KFStep(name="MakeResourceIntensive", category="Flow", toolTipText="Makes downstream connected steps resource intensive (or not). This shifts processing of such steps between the main step executor<br>service and the high resource executor service or vice versa.", iconPath="weka/gui/knowledgeflow/icons/DiamondPlain.gif") public class MakeResourceIntensive extends BaseStep
A Step that makes downstream steps that are directly connected to this step resource intensive (or not). This overrides whatever the downstream step may (or may not) have declared in it's `KFStep` class annotation with regards to whether it is resource intensive (cpu or memory). The Knowledge Flow execution environment uses two executor services - a primary one to execute batch processing for steps; and a secondary one for executing `StepTask`s (which are assumed to be resource intensive by default) or for executing batch processing for a Step when it declares itself to be resource intensive. This secondary executor service uses a limited (typically `<= num cpu cores`) number of threads. Steps that involve potentially intensive (cpu/memory) processing should declare themselves resource intensive so that less taxing steps (and the UI) get cpu cycles. E.g. the Classifier Step is resource intensive so that processing cross-validation folds in parallel for a large data set or computationally intensive classifier does not blow out memory or bog the system down.

Version:
  $Revision: $
Author:
  Mark Hall (mhall{[at]}pentaho{[dot]}com)
See Also:
  - Serialized Form

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `MakeResourceIntensive()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `List<String>`
  `getIncomingConnectionTypes()`
  Get a list of incoming connection types that this step can accept.
  `boolean`
  `getMakeResourceIntensive()`
  Get whether downstream steps are to be made resource intensive
  `List<String>`
  `getOutgoingConnectionTypes()`
  Get a list of outgoing connection types that this step can produce.
  `void`
  `processIncoming(Data data)`
  Process incoming data
  `void`
  `setMakeResourceIntensive(boolean resourceIntensive)`
  Set whether downstream steps are to be made resource intensive or not
  `void`
  `stepInit()`
  Initialize the step

  ### Methods inherited from class weka.knowledgeflow.steps.BaseStep

  `environmentSubstitute, getCustomEditorForStep, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, outputStructureForConnectionType, outputStructureForConnectionType, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop`

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### MakeResourceIntensive

    public MakeResourceIntensive()

- ## Method Details

  - ### setMakeResourceIntensive

    @OptionMetadata(displayName="Make downstream step(s) high resource", description="<html>Makes downstream connected steps resource intensive (or not)<br>This shifts processing of such steps between the main step executor service and the high resource executor service or vice versa.</html>") public void setMakeResourceIntensive(boolean resourceIntensive)
    Set whether downstream steps are to be made resource intensive or not
    Parameters:
      `resourceIntensive` - true if the downstream connected steps are to be made resource intensive

  - ### getMakeResourceIntensive

    public boolean getMakeResourceIntensive()
    Get whether downstream steps are to be made resource intensive
    Returns:
      true if downstream connected steps are to be made resource intensive

  - ### stepInit

    public void stepInit() throws WekaException
    Initialize the step
    Throws:
      `WekaException` - if a problem occurs

  - ### getIncomingConnectionTypes

    public List<String> getIncomingConnectionTypes()
    Get a list of incoming connection types that this step can accept. Ideally (and if appropriate), this should take into account the state of the step and any existing incoming connections. E.g. a step might be able to accept one (and only one) incoming batch data connection.
    Returns:
      a list of incoming connections that this step can accept given its current state

  - ### getOutgoingConnectionTypes

    public List<String> getOutgoingConnectionTypes()
    Get a list of outgoing connection types that this step can produce. Ideally (and if appropriate), this should take into account the state of the step and the incoming connections. E.g. depending on what incoming connection is present, a step might be able to produce a trainingSet output, a testSet output or neither, but not both.
    Returns:
      a list of outgoing connections that this step can produce

  - ### processIncoming

    public void processIncoming(Data data) throws WekaException
    Process incoming data
    Specified by:
      `processIncoming` in interface `BaseStepExtender`
    Specified by:
      `processIncoming` in interface `Step`
    Overrides:
      `processIncoming` in class `BaseStep`
    Parameters:
      `data` - the data to process
    Throws:
      `WekaException`