Skip to content
Snippets Groups Projects
Commit ecc8e26e authored by Armin Costa's avatar Armin Costa
Browse files

Update Features.md

parent beb7fae8
No related branches found
No related tags found
No related merge requests found
# Features # # Features #
## Concurrent Distribution of Data ## ## Concurrent Distribution of Data ##
- supported protocols: SFTP, FTP - supported protocols: SFTP, FTP
- recursive upload of directories - recursive upload of directories
- Notification of data delivery via e-mail - Notification of data delivery via e-mail
```xml ```xml
<AOI name="aoi" active="yes"> <AOI name="aoi" active="yes">
<Distribution type="sftp" active="no" priority="NORM"> <Distribution type="sftp" active="no" priority="NORM">
...@@ -28,11 +31,14 @@ ...@@ -28,11 +31,14 @@
``` ```
## Concurrent Download of Data ## ## Concurrent Download of Data ##
- supported protocols: SFTP, FTP - supported protocols: SFTP, FTP
- Notification of data received via e-mai - Notification of data received via e-mai
```xml ```xml
<AOI name="ReceivingStation" active="yes"> <AOI name="ReceivingStation" active="yes">
<Download type="sftp" active="yes" priority="NORM"> <Download type="sftp" active="yes" priority="NORM">
...@@ -44,4 +50,124 @@ ...@@ -44,4 +50,124 @@
</consumer> </consumer>
</Download> </Download>
</AOI> </AOI>
``` ```
\ No newline at end of file
## Concurrent Execution of Tasks ##
- Plugins in form of java classes, shell scripts/commands, perl scripts
- Task execution is driven and triggered by filtered FileType's
- On the fly java code compilation is supported
```xml
<AOI name="aoi" active="yes">
<Task name="renameFiles" type="class" active="yes" priority="NORM">
<Command>class</Command>
<FileType>SAMPLE.*.AA.txt</FileType>
<Parameter>edu.eurac.distributionserver.tasks.generic.FileRename
</Parameter>
<FilePrefix>RENAME</FilePrefix>
</Task>
<Task name="testCmd" type="cmd" active="no" priority="NORM">
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>false</NotifyEmail>
<Command>ping</Command>
<Parameter>127.0.0.1</Parameter>
</Task>
<Task name="testPerl" type="sh" active="no" priority="NORM">
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>false</NotifyEmail>
<Command>perl</Command>
<Parameter>./test_data/scripts/testPerl.pl</Parameter>
<Parameter>$CWD</Parameter>
</Task>
<Task name="moveToStorage" type="shell" active="no" priority="NORM">
<RunOnTrigger>/tmp/RunMoveToStorage</RunOnTrigger>
<Command>sh</Command>
<Parameter>./test_data/scripts/moveToStorage.sh</Parameter>
<Parameter>/raid0/NISDS2/raid/pub/gsfcdata/npp/viirs/level2/*
</Parameter>
<Parameter>/raid0/level2STORAGE/</Parameter>
</Task>
</AOI>
```
## Concurrent Parallel/Serial Execution of Task Groups ##
- Task groups can be **serial** or **parallel**
```xml
<AOI name="aoi" active="yes">
<TaskGroup name="testGroup" type="parallel" active="yes"
priority="NORM">
<Task name="testCmd1" type="cmd" active="yes" priority="NORM">
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>false</NotifyEmail>
<Command>ping</Command>
<Parameter>127.0.0.1</Parameter>
</Task>
<Task name="testCmd2" type="cmd" active="yes" priority="NORM">
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>false</NotifyEmail>
<Command>ping</Command>
<Parameter>127.0.0.1</Parameter>
</Task>
</TaskGroup>
<TaskGroup name="testGroup2" type="serial" active="no"
priority="NORM">
<Task name="testCmd1" type="cmd" active="yes" priority="NORM">
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>false</NotifyEmail>
<Command>ping</Command>
<Parameter>127.0.0.1</Parameter>
</Task>
<Task name="testCmd2" type="cmd" active="yes" priority="NORM">
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>false</NotifyEmail>
<Command>ping</Command>
<Parameter>127.0.0.1</Parameter>
</Task>
</TaskGroup>
</AOI>
```
## Concurrent Cleaning of data ##
- implements all switches available in linux find command
- On Windows OS a CyWIN installation is required
```xml
<AOI name="cleanSATWorkspaces" active="yes">
<DataCleaner name="01_Data" type="filesystem" active="yes"
priority="NORM">
<RunOnTrigger>/tmp/RunSATWS_cleanup</RunOnTrigger>
<EMail>support.rs@eurac.edu</EMail>
<NotifyEmail>yes</NotifyEmail>
<IncludePath>/mnt/SATWS/*/01_Data/*</IncludePath>
<ExcludeFileType>*.pro</ExcludeFileType>
<ExcludeFileType>*.java</ExcludeFileType>
<ExcludeFileType>*.f</ExcludeFileType>
<ExcludeFileType>*.m</ExcludeFileType>
<ExcludeFileType>*.r</ExcludeFileType>
<ExcludeFileType>*.c</ExcludeFileType>
<ExcludeFileType>*.h</ExcludeFileType>
<ExcludeFileType>*.doc</ExcludeFileType>
<Size>+1k</Size>
<LastAccess>+60</LastAccess>
</DataCleaner>
</AOI>
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment