Configuration.md

# Configuration #


    
## Main DES configuration file ##
 
 
This is the main DES configuration that contains basic configuration variables that are loaded at DES startup or restart

**DES.ini:** ./config/DES.ini  
 
 
  
### Sample configuration
 
  
```
[System]
RDS_ID=DES_YOURIDC@yourdomain.org
RDS_HOME=/DES
CONFIG_FILE_PATH=./test_data/config/
LOGS_PATH=LOG4J
BASE_PATH=./test_data/data/
STAMPS_PATH=./test_data/stamps
COMPILECODE_PATH=./test_data/tmp
MAX_PROCESSOR_THREADS=50
MAX_AOI_THREADS=50
MAX_AOI_TASKGROUP_THREADS=50
AOI_THREAD_SLEEP=10000
MAX_NR_FILTER_FILES=5
DEBUG=true
STDOUT_STDERR=false
[DB]
USE_STATUS_DB=false
DB_SERVER=server.yourdomain
DB_DATABASE=des
DB_PORT=5432
DB_USER=user
DB_PWD=pwd
[Mail]
FROM_EMAIL=des@yourdomain.org
SMTP=mailsubmit.yourdomain.org
SMTP_PORT=25
[Plugin]
CONFIG_PLUGIN=./plugins/plugin.ini
```
 
 
  
### Parameters ###
 
  
[System]  
**RDS_ID** DES identifier (ex. DES_$ID@yourdomain.org)  
**RDS_HOME** HOME directory of DES  
**CONFIG_FILE_PATH** Path where processor configuration files are located ( $PROCESSOR.processor_config.xml)  
**LOGS_PATH** STOUT print to stdout | LOG4J : use log4j2| Path of the log file dir (ending with / , ex: ./testdata/logs/ ) directory will be created  
**BASE_PATH** Basepath at which logical directory tree starts: $BASE_PATH/$AoiName/$ProcessorName  
**STAMPS_PATH** Directory for Stamp files  
**COMPILECODE_PATH** Temporary directory where class files are compiled on the fly. Content will be deleted at every restart  
**MAX_PROCESSOR_THREADS** Max processor Threads  
**MAX_AOI_THREADS** Max Threads per AOI  
**MAX_AOI_TASKGROUP_THREADS** Max Threads per TaskGroup  
**AOI_THREAD_SLEEP** Time interval for AOI Thread   
**MAX_NR_FILTER_FILES** Max number of files filtered in each iteration  
**DEBUG** Write to log files  
**STDOUT_STDERR** Print Tasks output streams to STDOUT/STDERR)  
[DB]  
**USE_STATUS_DB** Enable Task status logging in DB  
**DB_SERVER** = Database server  
**DB_DATABASE** = Database name  
**DB_PORT** = Database port  
**DB_USER** = Database user  
**DB_PWD** Database password  
[Mail]  
**FROM_EMAIL** e-mail for delivery  
**SMTP** = URL for SMTP  
**SMTP_PORT** Port for SMTP   
[Plugin]  
**CONFIG_PLUGIN** Configuration file for Plugins. Java class variable: DistributionServerConfig.CONFIG_PLUGIN    
 
 
    
## Authentication configuration file ##
 
 
Authentication parameters used in the operations **<Distribution>** and **<Download>** are stored within the file :

**DistributionAuth.xml:** ./config/DistributionAuth.xml
 
 
Sample entry:

```xml
<Auth>
	<AuthRef name="testuser@server.yourdomain" active="yes">
		<Host>server.yourdomain</Host>
		<User>testuser</User>
		<Pwd>test</Pwd>
		<Param>nativejava</Param>
	</AuthRef>
</Auth>
```
 
In the DES processor operation declarations the authentication entry can be references as follows: 

```xml
<consumer name="testuser@server.yourdomain" active="yes">
	<ProductType>*.txt</ProductType>
	<EMail>notifymail@yourdomain</EMail>
	<NotifyEmail>false</NotifyEmail>
	<Recursive>true</Recursive>
</Consumer>
```
  
## Processor configuration file
  
  
The DES executes concurrently a given set of *Processor* definition files that are defined in XML format.
A *Processor* configuraiton file ( *.processor_config.xml ) is the main executable entity for the DES. 
 
- There can be 1 - n *Processor* configuration files running on a given DES instance. 
- Each processor configuration file can have 1 - n *AOI* (Areas Of Interest).
- Each *AOI* can have 1 - n functional entities (*Distribution, Download, DataCleaner, Task, TaskGroup*) 
- The same xml format is used also by EGPF (Eurac Generic Processor Framework)

The *Processor* configuration options are listed here:

https://gitlab.inf.unibz.it/css-public/des_documentation/edit/master/Processors.md

  
## Start a DES instance ##
 
 
1.	Edit the ./config/DistributionServer.ini to fit your environment
2.	Create the set of *.processor_config.xml files that you want to run and place them in the path CONFIG_FILE_PATH as defined in DistributionServer.ini config file 
3.	Choose a suitable operation mode
 
print program arguments:
```
java -jar DES.jar –h
```
 
 
 
### Operation modes ###
  
  
1) Start a DES instance with default configuration defined in DES.ini and run all Processors (*.processor_config.xml files) located in CONFIG_PATH 
 
```
java -jar DES.jar
```
 
  
2) Start a DES instance with default configuration defined in DES.ini and run a single Processor
 
```
java -jar DES.jar -p yourtest.processor_config.xml
```
 
  
3) Start a DES instance with a different configuration (eg. DES_test.ini) and run all Processors (*.processor_config.xml files) located in CONFIG_PATH
 
```
java -jar DES.jar -c ./config/DES_test.ini
```
 
  
4) Start a DES instance with a different configuration (eg. DES_test.ini) and run a given Processor
 
```
java -jar DES.jar -c ./config/DES_test.ini -p yourtest.processor_config.xml
```
 
 
### Passing command line startup variables to be referenced in DES Processor  ###  
 
 
Custom arguments can be passed to the startup command line

```
java -jar DES.jar -v HOSTNAME=des.yourdomain -v IP=192.168.2.2
```

The defined variables can than be referenced in a given Processor definition with the '@' reference (eg. @HOSTNAME)

```xml
<Task name="testCmd" type="cmd" active="no" priority="NORM">
	<EMail>support.rs@eurac.edu</EMail>
	<NotifyEmail>false</NotifyEmail>
	<Command>ping</Command>
	<Parameter>@HOSTNAME</Parameter>
</Task>
```