Howtos.md

# Howto's

## Create a Trigger for a Task ##
 
 
There are different methods to trigger a *Task* execution process:
- Using the tag *RunOnTrigger*
- Using the tags *SartDate* and *StopDate*
- Using the filtering tag *FileType*

The most flexible method is actually the *FileType* filter, as everything is treated as a file being processed
 
 
 
### Using *FileType* filter method ###
 
 
The presence of a dedicated trigger file (eg. TR$PROCESSOR.*.trigger) , or also any other file, filtered by the <FileType> expression,
acts as a driver for the Task execution. This allows than to create a list of trigger files that the DES can then elaborate in a FIFO queue.

Because the Date for example is often used as input parameter for a processing time series for example,
the Date has to be passed as input parameter to the Task executing a particular job.

One approach to implement this, is to ensure that the trigger file TR$PROCESSOR.*.trigger includes the date at a certain position, i.e position 5 (. separated):
```
TREURAC_RADBEAM_P.southtyrol_highres.6120_6120.hour.8.2017-02-10T15:00:00.Eurac.01.00.trigger
```

So the implemented wrapper Script (*Perl*, *Python*, *Shell*) or Java Class for the Task can extract the processing parameters as needed, i.e. The date to be processed. 
The trigger file itself might contain even a more structured meta information in form of XML for example.

By convention the trigger file should have the following naming convention:
```
TR$PROCESSOR.$AOI.TileID.Type.Value.Date.Eurac.$versionMajor.$versionMinor.trigger
```
Any other naming convention can be adapted by providing an appropriate wrapper script that is called.

#### Example ####

**Processor configuration:**
```xml
<TaskGroup name="Processing" type="serial" active="yes" priority="NORM">
	<Task name="createMap" type="cmd" active="yes" priority="NORM">
		<EMail>mail@host.yourdomain</EMail>
		<NotifyEmail>OnError</NotifyEmail>
		<FileType>TREURAC_RADBEAM_P.*.trigger</FileType>
		<Command>sh</Command>
		<Parameter>/Algorithms/runProc.sh</Parameter>
	</Task>
	<Task>
		...
	</Task>
</TaskGroup>
```
 
 
**Task implementation:** (*runProc.sh*)
```sh
#!/bin/sh
arg=($*)
BASE_PATH=$1
AOI_NAME=$2
PROCESSOR_NAME=$3
TRIGGER_FILE_NAME=$4
 
echo "creating environment vars.."
# User specific aliases and functions
export PATH=$PATH:/Algorithms/
 
echo "start processing..."
 
file=$TRIGGER_FILE_NAME
echo "  processing file: $file"
 
DATE_FROM_FILE=$(echo "$file" | cut -d '.' -f 6)
echo "date: $DATE_FROM_FILE"
 
DATE=`date -d "$DATE_FROM_FILE"  +"%Y-%m-%d"`
HOUR=`date -d "$DATE_FROM_FILE" +"%H"`
HOUR2=$HOUR
HOUR=`expr $HOUR2 + 1`
MIN=`date -d "$DATE_FROM_FILE" +"%M"`
 
echo "running algorithm for date: $DATE"
```
 
 
Note: The Date format contained in the Trigger file naming convention should comply if possible with the format *yyyymmddThhmmss*

The trigger files may than be generated manually or via a script called by a Cronjob (see example below):
```
/Cronjobs/createDESTaskTriggers.sh /raid0/DES/data southtyrol_highres EURAC_RADBEAM_P 6120_6120 [ 20170101T091500 ]
```
 
 
**Cronjob Script:** (*createDESTaskTriggers.sh*)
```sh
#!/bin/sh
arg=($*)
BASE_PATH=$1
AOI_NAME=$2
PROCESSOR_NAME=$3
TILE_ID=$4
 
echo `date`
echo "creating trigger..."
TRIGGER_PATH=$BASE_PATH/$AOI_NAME/$PROCESSOR_NAME
DATE=`date +"%Y-%m-%dT%H:%M:%S"`
echo "Date: $DATE"
TR_FILE="TR$PROCESSOR_NAME.$AOI_NAME.$TILE_ID.year.2017.$DATE.Eurac.01.00.trigger"
echo "creating trigger $TRIGGER_PATH/$TR_FILE"
touch $TRIGGER_PATH/$TR_FILE
touch $TRIGGER_PATH/$TR_FILE.md5
echo "done"
echo "------------"
```
 
 
 
## Run a DES instance in Test Mode ##
 
When testing a new processor it could be useful to have a test environment where it is possible to check the output.
To do this is straightforward because for each installed DES instance there is a test_data folder where test can be performed.

**Steps:**
- Create a symbolic link to your processor: ln -s DES.jar DES_test.jar to have a better control on your DES_test (if you decide to call it in this way) processor
- Copy your YOUR_PROCESSOR.processor_config.xml file in the folder */DES/test_data/processor_config/*
- Do your changes to this file. For example you can enable or disable *AOI*, *Task*, *TaskGroup*, *Distribution*, *Download*, etc.
- Open the file */DES/config/DistributionServerTest.ini* and change the value of *LOGS_PATH* variable from *LOGS_PATH=LOG4J* to *LOGS_PATH=STDOUT*. This will allow to have the log in the standard output to facilitate the debug.
- Change to the DES folder: *cd /DES*
- Run the following command: 
```
java -jar DES_test.jar -c ./config/DistributionServerTest.ini -p YOUR_PROCESSOR.processor_config.xml
```