Skip to content
Snippets Groups Projects
Conventions.md 2.04 KiB
Newer Older
Armin Costa's avatar
Armin Costa committed
# Conventions #
 
 
Armin Costa's avatar
Armin Costa committed
## Logical directory hierarchy ##
 
Armin Costa's avatar
Armin Costa committed
The DES starts it's operations (eg. <FileType> filtering), considering a logical hierachy of directories starting form a given BASE_PATH that 
reflects the configuration within a given processor configuration file.

The directories are created automatically with the following structure:
```
$BASE_PATH/$AOI/$PROCESSOR
```
Armin Costa's avatar
Armin Costa committed
 
 
 
## FileType filtering ##
 
The files to be handled by a give *Task* or that are used for *Task* triggering, are filtered by regular expressions defined in the *FileType* tag of the *Task*

Files are detected by the *FileType* filtering IFF:
- An equivalend .md5 file exists in the logical directory hierarchy (ex.yourfile.txt.md5), unless the tag *MD5Filter* is set explicitly to value *false* 
- No stamp file exists (eg. yourfile.txt.stamp) in the path defined by *STAMPS_PATH/$PROCESSOR*

Stamp files are used to indicate that a file has already been processed by some Task

A given Task,

IFF a given *Task* returns either *EXIT_SUCESS* (0) or *EXIT_WARNING* (1), the DES will create an equivalent stamp file for the file processed

IF a given *Task* does not comply to the *EXIT_CODES* defined above, the *Task* implementation, upon completion,
should therefore either delete the input file and it's equivalent md5, create an equivalent stamp file in *STAMPS_PATH/$PROCESSOR* directory
 
 
 
## Stamp files ##
 
A Stamp file is an empty file that is created iff a given Task has been completed for a given input file filtered by a *FileType* expression.
In presence of a stamp file a subsequent *FileType* filtering will not detect the file that has an equivalent stamp file.

Stamp files are stored in the directory specified by *STAMPS_PATH/$PROCESSOR*

Stamp files have the following naming convention:
```
[PUSH | GET | TASK]_[Consumer name | Task name]_[Procesor name]_[AOI name]_[File name].stamp
```
 
Example:
```
TASK_createInstantSolarRadiation_EURAC_RADBEAM_P_southtyrol_TREURAC_RADBEAM_P.southtyrol.dem_ST_25m_LIDAR_32632.year.2017.2018-06-19T16:45:00.Eurac.01.00.trigger.stamp
```
Armin Costa's avatar
Armin Costa committed