Processors.md

 
# Processor #


  
## AOI ##
 
The *AOI* entity represents a logial hirarchy in a processor definition that is mapped also in the DES folder structure.

Each processor can have 1 -n *AOI*, also with the same name if required.
 
```xml
<EURAC_GENERIC_P>
	<Processing>
		<AOI name="yourAOI" active="yes">
			...
			...
		</AOI>
	</Processing>
</EURAC_GENERIC_P>
```

 
### Attributes ###
 
**name** = : Name of the AOI

**active** = yes | no : Enable or disable AOI

[**sleepFactor**] = positive int : Factor that multiplies the AOI_THREAD_SLEEP, namely the sleeping interval for the threads inside AOIs. This enables to have AOIs where Threads have bigger sleeping intervals 

 
### Tags ###
 
[[Distribution] [Download] [Task] [TaskGroup] [DataCleaner]] (1 - n)

  
## Distribution ##
 
The *Distribution* entity provides file transfer functionality.  
 
```xml
<Distribution type="ftp" active="no" priority="NORM">
	<consumer>
	</consumer>
</Distribution>
```
 
### Attributes ###
 
**type** = ftp | sftp : Transfer protocol

**active** = yes | no : Enable or disable Distribution

**priority** = MIN | NORM | MAX : Thread priority

 
### Tags ###
 
Consumer (1 - n)

  
  
## Download ##
 
Download
 
```xml
<Download type="sftp" active="yes" priority="NORM">
	<consumer>
	</consumer>
</Download>
```
 
### Attributes ###
 
**type** = sftp : Transfer protocol

**active** = yes | no : Enable or disable Distribution

**priority** = MIN | NORM | MAX : Thread priority

 
### Tags ###
 
Consumer (1 - n)

  
  
## Consumer ##
 
Consumer
 
```xml
<consumer name="user@host.yourdomain" active="yes">
	<ProductType>*.txt</ProductType>
	<EMail>mail@yourdomain</EMail>
	<NotifyEmail>false</NotifyEmail>
	<Recursive>true</Recursive>
</consumer>
```
 
### Attributes ###
 
**name** = : Name reference for authentication in DistributionAuth.xml

**active** = yes | no : Enable or disable Consumer

NOTE: It is worthwhile noting that in this file each name reference should be different from the others. It is strongly suggested to use a naming convention such as user@hostname as indicated in the follow: 
```xml
<AuthRef name="user@host.yourdomain" active="yes">
	<Host>host.yourdomain</Host>
	<User>user</User>
	<Pwd>pass</Pwd>
	<Param></Param>
</AuthRef>
```
 
### Tags ###
 
**RunOnTrigger** = : Absolute path of file that serves as trigger (has priority over StartDate/StopDate and overrides Runnable state). Trigger file will be deleted IFF ExitCode == 0 || ExitCode == 1 

**StartDate** = Date Time to start the Task (YYY-MM-DDTmm:hh:ss)

**StopDate** = Date Time to stop the Task (YYY-MM-DDTmm:hh:ss)

**FileType** (1 - n) = : regular expresison for file type filtering (default: *.*)

**MD5Filter** = true | false : Filters files defined in FileType/ProductType with check on exiting MD5 file. NOTE: for Downloads this can be set to 'false' whenever the server side file does not have an equivalent md5 file. For Distributions and Tasks it is advisable to have the default 'true' value in order to have a file completion validation, else the dataflow logic needs to guarantee file completion (default: true) 

**MD5Generation** = true | false : If not present, generates the MD5 file. NOTE: <MD5Filter> must be false in this case in order to have effect 

**MD5Check** = true | false: IFF <Download> AND <MD5Filter>==true recomputes the md5 checksum and compares to .md5 file 

**EMail** (1 -n ) = : Email notification recipient

**NotifyEmail** = true (OnError) | OnSuccess | OnError | OnWarning | false : To enable e-mail notifications on data delivery 

**RemoteDestinationPath** = : Path for directory in destination Host (default: $AOI/$PROCESSOR ) [ Deprecated keyword 'DestinationPath' is still valid ] 

**LocalDestinationPath** = : Path for directory local Host (default: $BASEPATH/$AOI/$PROCESSOR)

**ProductRename** = : File name delivered to remote Host (default: filname as filtered by FileType) 

**Recursive** = true | false : Enable recurive filtering and upload of data in directory hierarchies. Currently only applicable to <Distribution> 


## DataCleaner ##
 
DataCleaner
 
```xml
<DataCleaner name="HOME" type="filesystem" active="yes" priority="NORM">
	<EMail>mail@host.yourdomain</EMail>
	<NotifyEmail>yes</NotifyEmail>
	<IncludePath>/raid0/abz01rstest.eurac.edu/AdminCoA/*</IncludePath>
	<ExcludePath>/raid0/abz01rstest.eurac.edu/AdminCoA/doNOTdelete/*</ExcludePath>
	<ExcludeFileType>*.java</ExcludeFileType>
	<Size>+1k</Size>
	<LastAccess>+1</LastAccess>
</DataCleaner>
```
 
### Attributes ###

**name** = : Name of DataCleaner

**type** = filesystem : Type of data Cleaner

**active** = yes | no : Enable or disable DataCleaner

**priority** = MIN | NORM | MAX : Thread priority
 
 
### Tags ###
 
**RunOnTrigger** = : Absolute path of file that serves as trigger (has priority over StartDate/StopDate and overrides Runnable state). Trigger file will be deleted IFF ExitCode == 0 || ExitCode == 1 

**StartDate** = Date Time to start the Task (YYY-MM-DDTmm:hh:ss)

**StopDate** = Date Time to stop the Task (YYY-MM-DDTmm:hh:ss)

**EMail** (1 -n ) = : Email notification recipient

**NotifyEmail** = true (OnError) | OnSuccess | OnError | OnWarning | false : To enable e-mail 

**IncludePath** (1 - n) = : Path to be included in the cleaning policy

**ExcludePath** (1 - n) = : Path to be excluded in the cleaning policy

**ExcludeFileType** (1 - n) = : Regular expression for file types to be excluded from the cleaning policy 

**Size** = +int ( +1c, +1k, +1M, +1G) : Size in byte, kilobyte, megabyte, gigabyte (default: MIN_SIZE = "+1000k")

**LastAccess** = +int : Nr of days (default: LAST_ACCESS = "30")

**LastModified** = : +int : Nr of days (default: LAST_MODIFIED = "+30"; // Days ago from the current date)

**Type** = **f** (file) | **d** (directory) | **l** (link) : Type of data, only a single value allowed (default: f)
  
NOTE: Only available on Linux/UNIX compatible environments OR Windows with CyWin