Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • commul/docker/clarin-dspace
1 result
Show changes
Commits on Source (2)
Hide whitespace changes
Inline Side-by-side
Showing
with 383 additions and 282 deletions
.gitignore
View file @ d3180f02
.*.swp
kubernetes/secrets/
dockerfiles/commul-customization/certs/*
dockerfiles/commul-customization/original-webpage/
......@@ -6,3 +7,6 @@ dockerfiles/commul-customization/init-dspace-dbs.sh
kubernetes/echo-server.yaml
kubernetes/maintenance-pod.yaml
kubernetes/test-postgres-deployment.yaml
kubernetes/secrets*
clarin-dev_eurac_edu.key
clarin_eurac_edu.key
CHANGELOG.md
View file @ d3180f02
......@@ -12,6 +12,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Removed
### Fixed
## [1.3] - 2018-09-05
### Added
- added robots.txt to the expected place and made sure that sitemap was properly generated (#49)
- added some python modules to the DSpace Dockerfile that are needed by DSpace healthchecks
### Changed
- adapted some configuration to preserve the real IP addresses of visitors (#35)
- added configuration to enable DSPace to send Java errors to us by mail
### Fixed
- Made sure that tomcat is stopped with `-force` in entrypoint script to ensure that only one instance is running
## [1.2.2] - 2018-08-31
### Fixed
- Changed the DSpace config to use the production handle prefix and not the staging one
## [1.2] - 2018-08-27
### Added
......@@ -60,3 +82,5 @@ This is the first release. It all starts here.
[Unreleased]:https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/compare/v1.1...HEAD
[1.1]: https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/compare/v1.0...v1.1
[1.2]: https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/compare/v1.1...v1.2
[1.2.2]: https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/compare/v1.2...v1.2.2
[1.3]: https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/compare/v1.2.2...v1.3
README.md
View file @ d3180f02
......@@ -8,90 +8,60 @@ You need a Linux server (we tested on Ubuntu 16.04 and CentOS 7.3) with a recent
None of the files contain passwords, instead passwords and other confidential information are provided to the containers using environment variables. In Kubernetes you can use [secrets](https://kubernetes.io/docs/concepts/configuration/secret/) for this.
# Exemplary workflow using Docker Compose
## get Dockerfiles
```
git clone https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/
cd clarin-dspace
```
# Kubernetes
## get my versions of dist files
```
cp password_mod.sh.dist password_mod.sh
cp commul-customization/init-dspace-dbs.sh.dist commul-customization/init-dspace-dbs.sh
cp commul-customization/local.properties.dist commul-customization/local.properties
```
## Prerequisites
## change passwords
```
vi password_mod.sh
chmod +x password_mod.sh
./password_mod.sh
```
* Kubernetes cluster
* Ceph filesystem
* GitLab including docker registry
* Kubernetes credentials
* a working [kubectl] (https://kubernetes.io/docs/tasks/tools/install-kubectl/, make sure the kubectl and your Kubernetes cluster are compatible versions)
## make sure the certificate and key are there
```
cp -r /tmp/certs ./commul-customization/
```
## Setup
## build the images
```
docker-compose up -d --build
```
All sensitive information is stored in Kubernetes secrets. If you want to use the Kubernetes yaml files as is, make sure that you create those secrets with the same secret and key names.
## enter the DSpace container
```
docker exec -ti clarindspace_dspace_1 bash
```
## deploy DSpace
```
make new_deploy
```
## Workflow
## copy over modified aai_config.js
```
cp /tmp/aai_config.js /opt/lindat-dspace/installation/webapps/xmlui/themes/UFAL/lib/js/
```
### Building docker images
## create dspace admin as tomcat8, so that the log files have the right owner
```
su -s /bin/sh tomcat8
/opt/lindat-dspace/installation/bin/dspace create-administrator
```
Kubernetes is pulling images from a registry, so you need to first build the docker files locally and push them to a registry. We are using the registry feature of GitLab. Make sure to set up your local docker so that it can upload images into the registry.
We try to keep the version numbers of all images in sync and use the script [release.sh](dockerfiles/release.sh) for this. You will need to edit this script and exchange the registry URL with your own. The script takes the new version number as its argument and builds all docker images and pushes them into our registry.
## start the dspace webapp
```
cd /opt/repository/sources/dspace/utilities/project_helpers/scripts
/etc/init.d/tomcat8 start
cd dockerfiles
./release 1.2.3
```
# Kubernetes
By default this is pushing images into the staging branch of the registry. If you want to build a production image, you need to provide the extra argument `production`.
## Prerequisites
* Kubernetes cluster
* Kubernetes credentials
* a working [kubectl] (https://kubernetes.io/docs/tasks/tools/install-kubectl/)
# Workflow
## Building docker images
Kubernetes is pulling images from a registry, so you need to first build the docker files. We try to keep the version numbers of all images in sync and use the script [release.sh](dockerfiles/release.sh) for this. It takes the new version number as its argument and builds all docker images and pushes them into our registry. You will need to edit this script and exchange the registry URL with your own.
```
cd dockerfiles
./release 1.2.3 production
```
## deploying to Kubernetes
### deploying to Kubernetes
After pushing all images to your container registry you can ask Kubernetes to pull and deploy them. If necessary, you might need to create a [personal token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) for logging into the GitLab Registry and store it as a secret that you then reference in the Kubernetes yaml files.
After pushing all images to your container registry you can ask Kubernetes to pull and deploy them. If necessary, you might need to create a [personal token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) for logging into the GitLab Registry and store it as a secret that you then reference in the Kubernetes yaml files. You need to edit the yaml files to suit your setup, especially you need to edit:
* The image registry URL
* The ceph setup
* The name of your Kubernetes namespace
There is a handy script that calls all necessary kubectl commands one after another called [start-kube.sh](kubernetes/start-kube.sh), for bringing it all down again use [stop-kube.sh](kubernetes/stop-kube.sh) instead. For this to work, make sure that you have the correct images referenced in the deployment yaml files.
These edits you only have to do once, but for each new deploy you have to edit the version numbers of the images in the deployment yaml files. This can be done with a simple sed:
```
image: gitlab.inf.unibz.it:4567/commul/docker/clarin-dspace/staging/dspace:1.2-RC1
sed -i 's/1.2-RC1/1.2/' ../kubernetes/*deploy*
```
There is a handy script that calls all necessary kubectl commands one after another called [start-kube.sh](kubernetes/start-kube.sh), for bringing it all down again use [stop-kube.sh](kubernetes/stop-kube.sh) instead. By default this doesn't touch the persistent volume claim. For a full redeploy that forgets all history you have to call the kubectl command manually:
```
kubectl delete -f pgdata-persistentvolumeclaim.yaml
```
......@@ -109,145 +79,61 @@ secrets
d
# Exemplary workflow using Docker Compose (this might be outdated)
### Use existing Docker image
```console
docker image ls -a gitlab.scientificnet.org:4567/commul/dspace/*
docker pull gitlab.scientificnet.org:4567/commul/dspace/<my_image>:<my tag>
## get Dockerfiles
```
### Build images
* build your docker image from `/clarin-dspace-docker` directory
* push to `https://gitlab.scientificnet.org/commul/dspace/container_registry`
```console
docker build -t gitlab.scientificnet.org:4567/commul/dspace/<my_image>:<my tag> -f Dockerfile.XX .
docker push gitlab.scientificnet.org:4567/commul/dspace/<my_image>:<my tag>
git clone https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/
cd clarin-dspace
```
# Installation
This step in a "workspace preparation" and is not directly related with dspace.
Provide gitlab & RBD credentials
```console
$ kubectl -n dspace-dev create -f gitlab-scientificnet-org-registry.yaml
## get my versions of dist files
```
```console
$ kubectl -n dspace-dev create -f rbd-secret.yaml
cp password_mod.sh.dist password_mod.sh
cp commul-customization/init-dspace-dbs.sh.dist commul-customization/init-dspace-dbs.sh
cp commul-customization/local.properties.dist commul-customization/local.properties
```
## PostgreSQL
To install PostgreSQL, we need some passwords, so we need Kubernetes secrets
```console
$ kubectl -n dspace-dev create -f postgresql-secret.yaml
secret "dspace-postgresql" created
## change passwords
```
```console
$ kubectl -n dspace-dev create -f dspace-secrets.yaml
secret "dspace-secrets" created
vi password_mod.sh
chmod +x password_mod.sh
./password_mod.sh
```
```console
$ kubectl -n dspace-dev get secrets
NAME TYPE DATA AGE
ceph-rbd-secret kubernetes.io/rbd 1 14m
default-token-c1pd4 kubernetes.io/service-account-token 3 4d
dspace-postgresql Opaque 1 1h
dspace-secrets Opaque 1 1m
gitlab-scientificnet-org-registry kubernetes.io/dockerconfigjson 1 46m
## make sure the certificate and key are there
```
Next, create a RBD Claim
```console
$ kubectl -n dspace-dev create -f postgresql-pvc.yaml
persistentvolumeclaim "dspace-postgresql" created
$ kubectl -n dspace-dev get pvc
NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE
dspace-postgresql Bound pvc-610c358d-a518-11e7-b21c-b60483de6a40 8Gi RWO auto 2s
cp -r /tmp/certs ./commul-customization/
```
Ok, now we are ready to deploy our PostgreSQL!
```console
$ kubectl -n dspace-dev create -f postgresql.yaml
$ kubectl -n dspace-dev get po
NAME READY STATUS RESTARTS AGE
dspace-postgresql-1806947503-cj077 0/1 ContainerCreating 0 2s
$ kubectl -n dspace-dev get po
NAME READY STATUS RESTARTS AGE
dspace-postgresql-1806947503-qw8q4 0/1 Running 0 9s
$ kubectl -n dspace-dev get po
NAME READY STATUS RESTARTS AGE
dspace-postgresql-1806947503-qw8q4 1/1 Running 0 46s
## build the images
```
docker-compose up -d --build
```
We have now launched the pod but we have to wait until the pod is up before accessing it
```console
$ kubectl -n dspace-dev exec -it dspace-postgresql-1806947503-qw8q4 bash
root@dspace-postgresql-1806947503-qw8q4:/#
## enter the DSpace container
```
docker exec -ti clarindspace_dspace_1 bash
```
Also, we can also test our PostgreSQL installation with a port forward
```console
$ kubectl -n dspace-dev port-forward dspace-postgresql-1806947503-qw8q4 5432:5432
## deploy DSpace
```
make new_deploy
```
now we are able to connect to localhost:5432
![pgAdmin_connect](pgadmin_1.png)
![pgAdmin_showTables](pgadmin_2.png)
```console
$ kubectl -n dspace-dev create -f postgresql-service.yaml
service "dspace-postgresql" created
## copy over modified aai_config.js
```
```console
$ kubectl -n dspace-dev exec -it servlet env | grep POSTGRESQL
DSPACE_POSTGRESQL_SERVICE_PORT_POSTGRESQL=5432
DSPACE_POSTGRESQL_PORT_5432_TCP_ADDR=10.13.0.173
DSPACE_POSTGRESQL_SERVICE_HOST=10.13.0.173
DSPACE_POSTGRESQL_PORT_5432_TCP_PROTO=tcp
DSPACE_POSTGRESQL_PORT_5432_TCP_PORT=5432
DSPACE_POSTGRESQL_SERVICE_PORT=5432
DSPACE_POSTGRESQL_PORT=tcp://10.13.0.173:5432
DSPACE_POSTGRESQL_PORT_5432_TCP=tcp://10.13.0.173:5432
cp /tmp/aai_config.js /opt/lindat-dspace/installation/webapps/xmlui/themes/UFAL/lib/js/
```
## DSpace
```console
$ kubectl -n dspace-dev create -f servlet.yaml
pod "servlet" created
## create dspace admin as tomcat8, so that the log files have the right owner
```
```console
$ kubectl -n dspace-dev get po
NAME READY STATUS RESTARTS AGE
dspace-postgresql-1806947503-qw8q4 1/1 Running 0 4d
servlet 1/1 Running 0 34s
su -s /bin/sh tomcat8
/opt/lindat-dspace/installation/bin/dspace create-administrator
```
```console
$ kubectl -n dspace-dev logs servlet
[Wed Oct 4 12:24:25 UTC 2017] Starting
[Wed Oct 4 12:24:25 UTC 2017] Cloning lindat-dspace repository...skip! /opt/repository/sources/dspace already exists
[Wed Oct 4 12:24:25 UTC 2017] Creating workspace...skip! /opt/repository/workspace already exists
[Wed Oct 4 12:24:25 UTC 2017] Copy makefile...skip! /opt/repository/workspace/config/variable.makefile already exists
[Wed Oct 4 12:24:25 UTC 2017] Copy local.properties...skip! /opt/repository/workspace/sources/local.properties already exists
[Wed Oct 4 12:24:25 UTC 2017] Compilation and Deployment of DSpace...already done
already done
already done
[Wed Oct 4 12:24:25 UTC 2017] Copy aai_config.js...skip! /opt/lindat-dspace/installation/webapps/xmlui/themes/UFAL/lib/js/aai_config.js already exists
[Wed Oct 4 12:24:25 UTC 2017] Create dspace admin...already done
[Wed Oct 4 12:24:25 UTC 2017] Starting Tomcat...Starting Tomcat 8...
Tomcat started.
## start the dspace webapp
```
```console
$ kubectl -n dspace-dev port-forward servlet 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
cd /opt/repository/sources/dspace/utilities/project_helpers/scripts
/etc/init.d/tomcat8 start
```
connect to http://localhost:8080/repository/xmlui/#
clarin-dev2clarin.sh
View file @ d3180f02
......@@ -6,10 +6,11 @@ perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/local.prope
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/default-ssl
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/aai.js
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/aai_config.js
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/aai_config.js
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/aa-statistics.php
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/clarin.eurac.edu.template.metadata.xml
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/webpage/index.html
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/commul-customization/webpage/index.html
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/Dockerfile.nginx
# adapt URL and prefix for handle server
perl -pi -e 's/20.500.12084/20.500.12124/g' dockerfiles/commul-customization/config.dct
......@@ -22,7 +23,6 @@ perl -pi -e 's/_5d15d5a81bc952da1d34986ebc7713c0d5c9ea61/_5d15d5a81bc952da1d3498
perl -pi -e 's/clarin-dev/clarin/g' dockerfiles/docker-compose.yml
echo "Modified all files to point to clarin.eurac.edu instead of clarin-dev.eurac.edu."
# adapt namespace and image pull path in kubernetes yaml files
sed -i 's/dspace-dev/dspace/' kubernetes/*.yaml
# adapt namespace in kubernetes yaml files
sed -i 's/dspace-dev/dspace/' kubernetes/*.yaml
perl -pi -e 's/clarin-dev/clarin/g' kubernetes/dspace-ingress.yaml
perl -pi -e 's#/staging/#/production/#g' kubernetes/*deploy*yaml
dockerfiles/Dockerfile.dspace
View file @ d3180f02
......@@ -68,6 +68,9 @@ RUN ./setup.sh /opt/repository/workspace
COPY commul-customization/variable.makefile /opt/repository/workspace/config/variable.makefile
# COPY commul-customization/dspace.cfg /opt/repository/sources/dspace/dspace/config/dspace.cfg
# install python modules: magic dateutil, lxml
RUN apt-get update && apt-get install -y python-magic python-dateutil python-lxml
# copy over configs and startup scripts
COPY commul-customization/start_stack.sh /opt/repository/workspace/scripts/
COPY commul-customization/stop_stack.sh /opt/repository/workspace/scripts/
......@@ -83,7 +86,7 @@ RUN make new_deploy
# DEBUG: to be able to access tomcat manager through browser
# COPY commul-customization/tomcat-users.xml /opt/tomcat8/conf/
RUN perl -pi -e 's/#xmlui\.session\.ipcheck = true/xmlui.session.ipcheck = false/' /opt/lindat-dspace/installation/config/dspace.cfg
RUN perl -pi -e 's/#useProxies = true/useProxies = true/' /opt/lindat-dspace/installation/config/dspace.cfg
# set up handle server
COPY commul-customization/hdl-setup.tmp /tmp/
......@@ -98,4 +101,3 @@ RUN perl -pi -e 's#DSPACE_INSTALLATION_DIRECTORY=#DSPACE_INSTALLATION_DIRECTORY=
COPY commul-customization/entrypoint.sh /usr/bin/entrypoint.sh
RUN chmod u+x /usr/bin/entrypoint.sh
ENTRYPOINT ["/usr/bin/entrypoint.sh"]
dockerfiles/Dockerfile.nginx
View file @ d3180f02
......@@ -49,6 +49,10 @@ RUN chmod a+x /etc/init.d/nginx
RUN ln -s /opt/nginx/sbin/nginx /usr/sbin/nginx
# copy over static html
COPY commul-customization/webpage/ /opt/nginx/html/
# copy over robots.txt
COPY commul-customization/robots-clarin.txt /opt/nginx/html/robots.txt
COPY commul-customization/google4a439c0ac1ac30d0.html /opt/nginx/html/
RUN chown -R www-data:www-data /opt/nginx/html/
#RUN mkdir /opt/nginx/html/img
#COPY commul-customization/index.html /opt/nginx/html/
......@@ -72,6 +76,8 @@ RUN /tmp/shibboleth_sp_with_fastcgi.sh
# copy the init script
RUN cp /opt/shibboleth-sp-fastcgi/etc/shibboleth/shibd-debian /etc/init.d/shibd
RUN chmod a+x /etc/init.d/shibd
# add IdP metadata from CLARIN server
ADD https://infra.clarin.eu/aai/prod_md_about_spf_idps.xml /opt/shibboleth-sp-fastcgi/var/cache/shibboleth/
# create the test secure folder and set up perl fastcgi
RUN mkdir /opt/nginx/html/secure
......@@ -110,11 +116,11 @@ WORKDIR /opt/repository/sources/lindat-aai-discovery/
# RUN make aai.min.js
RUN yui-compressor -o aai.min.js aai.js
# copy certificate for clarin-dev
# COPY commul-customization/certs/clarin-dev.key /etc/ssl/private/
# COPY commul-customization/certs/clarin-dev_eurac_edu.crt /etc/ssl/certs/
# copy certificate for clarin
# COPY commul-customization/certs/clarin.key /etc/ssl/private/
# COPY commul-customization/certs/clarin_eurac_edu.crt /etc/ssl/certs/
# if deployed on clarin instead of clarin-dev comment the two lines above and uncomment the following lines
# if deployed on clarin instead of clarin comment the two lines above and uncomment the following lines
## copy certificate for clarin
# COPY commul-customization/certs/clarin.key /etc/ssl/private/
......
dockerfiles/commul-customization/default-ssl
View file @ d3180f02
upstream tomcats {
server dspace:8009;
keepalive 10;
server dspace:8009;
keepalive 10;
}
server {
listen 80;
listen [::]:80;
server_name clarin.eurac.edu;
listen 80;
listen [::]:80;
server_name clarin.eurac.edu;
location /repository/oai {
location /repository/oai {
client_body_buffer_size 32K;
client_body_buffer_size 32K;
client_max_body_size 4G;
client_max_body_size 4G;
sendfile on;
send_timeout 300s;
sendfile on;
send_timeout 300s;
ajp_keep_conn on;
ajp_pass tomcats;
ajp_keep_conn on;
ajp_pass tomcats;
}
}
location / {
return 301 https://$server_name$request_uri;
}
location / {
return 301 https://$server_name$request_uri;
}
}
server {
......@@ -49,18 +49,20 @@ server {
add_header 'Access-Control-Allow-Credentials' 'true';
# redirect different legacy forms to repository/xmlui
rewrite ^/xmlui(.*)$ /repository/xmlui$1 redirect;
rewrite ^/repository.?$ /repository/xmlui/ redirect;
rewrite ^/solr(.*)$ /repository/solr$1 redirect;
rewrite ^/handle/(.*)$ /repository/xmlui/handle/$1 redirect;
# changed 'redirect' to 'permanent'
rewrite ^/xmlui(.*)$ /repository/xmlui$1 permanent;
rewrite ^/repository.?$ /repository/xmlui/ permanent;
rewrite ^/solr(.*)$ /repository/solr$1 permanent;
rewrite ^/handle/(.*)$ /repository/xmlui/handle/$1 permanent;
set $oai_index /repository/oai/request?verb=Identify;
rewrite ^/oai$ $oai_index redirect;
rewrite ^/oai$ $oai_index permanent;
location /oai {
return 301 https://$http_host/repository$request_uri;
}
rewrite ^/repository/oai$ $oai_index redirect;
rewrite ^/repository/oai/?$ $oai_index permanent;
# kept 'redirect' to be sure that this keeps working
if ($http_accept ~ "(.*xml.cmdi.*)"){
rewrite ^/(.*)/xmlui/handle/(.*)$ https://$http_host/repository/oai/cite?metadataPrefix=cmdi&handle=$2? redirect;
}
......@@ -73,26 +75,27 @@ server {
rewrite ^/(.*)/xmlui/handle/(.*)$ https://$http_host/repository/oai/cite?metadataPrefix=%1&handle=$2? redirect;
}
# location = / {
# return 301 /repository;
# }
# location = / {
# return 301 /repository;
# }
location / {
index index.php index.html index.htm;
# try_files $uri $uri/ /index.php?$args;
index index.html index.htm;
if ($HTTP_X_SCHEME = "http") {
rewrite ^ https://$http_host$request_uri permanent;
}
}
location /php/aa-statistics.php {
shib_request /shibauthorizer;
shib_request_use_headers on;
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
fastcgi_param SCRIPT_FILENAME /opt/nginx/html$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
include fastcgi_params;
}
shib_request /shibauthorizer;
shib_request_use_headers on;
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
fastcgi_param SCRIPT_FILENAME /opt/nginx/html$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
include fastcgi_params;
}
location /repository {
......@@ -133,20 +136,20 @@ server {
# add path your repository path that will be protected by shibboleth
location /repository/xmlui/shibboleth-login {
include repository_auth;
include repository_auth;
ajp_keep_conn on;
ajp_pass tomcats;
}
}
location /xmlui/shibboleth-login {
include repository_auth;
ajp_keep_conn on;
include repository_auth;
ajp_keep_conn on;
ajp_pass tomcats;
}
}
# remove the robots and favicon from the logs
location ~ /robots.txt$ { access_log off; log_not_found off; }
# location ~ ^/favicon.ico$ { access_log off; log_not_found off; }
location ~ /robots.txt$ { access_log off; }
location ~ ^/favicon.ico$ { access_log off; log_not_found off; }
# this prevents hidden files (beginning with a period) from being served
location ~ /\. { access_log off; log_not_found off; deny all; }
......@@ -170,17 +173,17 @@ server {
}
location /secure {
#include shib_clear_headers;
#Add your attributes here. They get introduced as headers
#by the FastCGI authorizer so we must prevent spoofing.
#more_clear_input_headers 'displayName' 'mail' 'persistent-id';
shib_request /shibauthorizer;
shib_request_use_headers on;
gzip off;
include shib_fastcgi_params;
fastcgi_pass unix:/tmp/fcgiwrap.socket;
fastcgi_param SCRIPT_FILENAME /opt/nginx/html$fastcgi_script_name;
}
# include shib_clear_headers;
# Add your attributes here. They get introduced as headers
# by the FastCGI authorizer so we must prevent spoofing.
# more_clear_input_headers 'displayName' 'mail' 'persistent-id';
shib_request /shibauthorizer;
shib_request_use_headers on;
gzip off;
include shib_fastcgi_params;
fastcgi_pass unix:/tmp/fcgiwrap.socket;
fastcgi_param SCRIPT_FILENAME /opt/nginx/html$fastcgi_script_name;
}
include /opt/nginx/conf/proxies-enabled/*;
}
dockerfiles/commul-customization/entrypoint.sh
View file @ d3180f02
......@@ -2,7 +2,7 @@
# replace environment variables in config files
perl -pi -e 's/\$\{DSPACE_USER\}/$ENV{DSPACE_USER}/; s/\$\{DSPACE_PASSWORD\}/$ENV{DSPACE_PASSWORD}/; s/\$\{MAIL_USER\}/$ENV{MAIL_USER}/; s/\$\{MAIL_PASSWORD\}/$ENV{MAIL_PASSWORD}/; s/\$\{DSPACE_DBNAME\}/$ENV{DSPACE_DBNAME}/; s/\$\{DSPACE_UTILSDBNAME\}/$ENV{DSPACE_UTILSDBNAME}/;' /opt/repository/sources/dspace/local.properties
perl -pi -e 's/\$\{DSPACE_USER\}/$ENV{DSPACE_USER}/; s/\$\{DSPACE_PASSWORD\}/$ENV{DSPACE_PASSWORD}/; s/\$\{MAIL_USER\}/$ENV{MAIL_USER}/; s/\$\{MAIL_PASSWORD\}/$ENV{MAIL_PASSWORD}/; s/\$\{DSPACE_DBNAME\}/$ENV{DSPACE_DBNAME}/; s/\$\{DSPACE_UTILSDBNAME\}/$ENV{DSPACE_UTILSDBNAME}/;' /opt/repository/sources/dspace/target/local.properties
perl -pi -e 's/\$\{DSPACE_USER\}/$ENV{DSPACE_USER}/; s/\$\{DSPACE_PASSWORD\}/$ENV{DSPACE_PASSWORD}/; s/\$\{MAIL_USER\}/$ENV{MAIL_USER}/; s/\$\{MAIL_PASSWORD\}/$ENV{MAIL_PASSWORD}/; s/\$\{DSPACE_DBNAME\}/$ENV{DSPACE_DBNAME}/; s/\$\{DSPACE_UTILSDBNAME\}/$ENV{DSPACE_UTILSDBNAME}/;' /opt/repository/sources/dspace/target/local.properties
perl -pi -e 's/\$\{DSPACE_USER\}/$ENV{DSPACE_USER}/; s/\$\{DSPACE_PASSWORD\}/$ENV{DSPACE_PASSWORD}/; s/\$\{MAIL_USER\}/$ENV{MAIL_USER}/; s/\$\{MAIL_PASSWORD\}/$ENV{MAIL_PASSWORD}/; s/\$\{DSPACE_DBNAME\}/$ENV{DSPACE_DBNAME}/; s/\$\{DSPACE_UTILSDBNAME\}/$ENV{DSPACE_UTILSDBNAME}/;' /opt/lindat-dspace/installation/config/dspace.cfg
perl -pi -e 's/\$\{DSPACE_USER\}/$ENV{DSPACE_USER}/; s/\$\{DSPACE_PASSWORD\}/$ENV{DSPACE_PASSWORD}/; s/\$\{MAIL_USER\}/$ENV{MAIL_USER}/; s/\$\{MAIL_PASSWORD\}/$ENV{MAIL_PASSWORD}/; s/\$\{DSPACE_DBNAME\}/$ENV{DSPACE_DBNAME}/; s/\$\{DSPACE_UTILSDBNAME\}/$ENV{DSPACE_UTILSDBNAME}/;' /opt/lindat-dspace/installation/config/modules/lr.cfg
......@@ -18,9 +18,8 @@ cd /opt/repository/workspace/scripts
make init_statistics
make update_discovery
make update_oai
/opt/tomcat8/bin/shutdown.sh
/opt/lindat-dspace/installation/bin/dspace generate-sitemaps
/opt/tomcat8/bin/shutdown.sh -force
# start tomcat
/opt/tomcat8/bin/catalina.sh run
dockerfiles/commul-customization/google4a439c0ac1ac30d0.html 0 → 100644
View file @ d3180f02
google-site-verification: google4a439c0ac1ac30d0.html
\ No newline at end of file
dockerfiles/commul-customization/local.properties
View file @ d3180f02
......@@ -20,7 +20,7 @@ lr.pid.community.configurations = community=*,prefix=20.500.12124,type=local,can
lr.pid.resolvemetadata = true
handle.canonical.prefix = http://hdl.handle.net/
handle.prefix = 20.500.12124
handle.prefix = 20.500.12124
handle.dir=${dspace.dir}/handle-server
lr.reserve.pid.on.start = false
......@@ -145,7 +145,7 @@ mail.registration.notify = ${mail.admin}
mail.admin = ${mail.feedback.recipient}
# Email where errors will be sent (from specific classes)
lr.errors.email=
lr.errors.email= appliedlinguisticsdevs@eurac.edu
# solr server address that your instance is using.
solr.server = http://localhost:8080/repository/solr
......
dockerfiles/commul-customization/nginx.conf
View file @ d3180f02
......@@ -19,37 +19,51 @@ http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
set_real_ip_from 10.12.0.0/16;
real_ip_header X-Forwarded-For;
#access_log logs/access.log main;
# equiv to the predefined "combined" format:
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent"';
sendfile on;
#tcp_nopush on;
access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
gzip on;
gzip_disable "msie6";
gzip_comp_level 6;
gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_proxied any;
# We try to compress on the far-back end and hope it will pass the ingress unharmed
# (and is still helpful)
gzip on;
gzip_comp_level 5;
gzip_http_version 1.1;
gzip_min_length 1024;
gzip_proxied any;
gzip_vary on;
gzip_disable "msie6";
gzip_types
text/plain
text/css
text/js
text/xml
text/javascript
application/atom+xml
application/javascript
application/x-javascript
application/json
application/xml
application/rss+xml
image/svg+xml;
application/vnd.ms-fontobject
application/x-font-ttf
application/x-web-app-manifest+json
application/xhtml+xml
application/xml
font/opentype
image/svg+xml
image/x-icon
text/css
text/javascript
text/js
text/plain
text/xml
text/x-component;
include default-ssl;
include default-ssl;
}
dockerfiles/commul-customization/robots-clarin-dev.txt 0 → 100644
View file @ d3180f02
User-agent: *
# Disable access
Disallow: /
dockerfiles/commul-customization/robots-clarin.txt 0 → 100644
View file @ d3180f02
# The FULL URL to the DSpace sitemaps
# The https://clarin.eurac.edu/repository/xmlui will be auto-filled with the value in dspace.cfg
# XML sitemap is listed first as it is preferred by most search engines
Sitemap: https://clarin.eurac.edu/repository/xmlui/sitemap
Sitemap: https://clarin.eurac.edu/repository/xmlui/htmlmap
##########################
# Default Access Group
# (NOTE: blank lines are not allowable in a group record)
##########################
User-agent: *
# Disable access to Discovery search and filters
Disallow: /repository/xmlui/discover
Disallow: /repository/xmlui/feedback
Disallow: /repository/xmlui/forgot
Disallow: /repository/xmlui/login
Disallow: /repository/xmlui/register
Disallow: /repository/xmlui/search-filter
Disallow: /Shibboleth.sso
#
# Optionally uncomment the following line ONLY if sitemaps are working
# and you have verified that your site is being indexed correctly.
# Disallow: /browse
#
# If you have configured DSpace (Solr-based) Statistics to be publicly
# accessible, then you may not want this content to be indexed
# Disallow: /statistics
#
# You also may wish to disallow access to the following paths, in order
# to stop web spiders from accessing user-based content
# Disallow: /contact
# Disallow: /feedback
# Disallow: /forgot
# Disallow: /login
# Disallow: /register
##############################
# Section for misbehaving bots
# The following directives to block specific robots were borrowed from Wikipedia's robots.txt
##############################
# advertising-related bots:
User-agent: Mediapartners-Google*
Disallow: /
# Crawlers that are kind enough to obey, but which we'd rather not have
# unless they're feeding search engines.
User-agent: UbiCrawler
Disallow: /
User-agent: DOC
Disallow: /
User-agent: Zao
Disallow: /
# Some bots are known to be trouble, particularly those designed to copy
# entire sites. Please obey robots.txt.
User-agent: sitecheck.internetseer.com
Disallow: /
User-agent: Zealbot
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: Fetch
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: TeleportPro
Disallow: /
User-agent: WebZIP
Disallow: /
User-agent: linko
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: Microsoft.URL.Control
Disallow: /
User-agent: Xenu
Disallow: /
User-agent: larbin
Disallow: /
User-agent: libwww
Disallow: /
User-agent: ZyBORG
Disallow: /
User-agent: Download Ninja
Disallow: /
# Misbehaving: requests much too fast:
User-agent: fast
Disallow: /
#
# If your DSpace is going down because of someone using recursive wget,
# you can activate the following rule.
#
# If your own faculty is bringing down your dspace with recursive wget,
# you can advise them to use the --wait option to set the delay between hits.
#
#User-agent: wget
#Disallow: /
#
# The 'grub' distributed client has been *very* poorly behaved.
#
User-agent: grub-client
Disallow: /
#
# Doesn't follow robots.txt anyway, but...
#
User-agent: k2spider
Disallow: /
#
# Hits many times per second, not acceptable
# http://www.nameprotect.com/botinfo.html
User-agent: NPBot
Disallow: /
# A capture bot, downloads gazillions of pages with no public benefit
# http://www.webreaper.net/
User-agent: WebReaper
Disallow: /
dockerfiles/commul-customization/webpage/images/clarin_ill_1100x375_2.png 0 → 100644
View file @ d3180f02
dockerfiles/commul-customization/webpage/images/clarin_ill_1100x375_2.png

14.7 KiB

dockerfiles/commul-customization/webpage/images/ercc-logo.png
View replaced file @ e962ee7d View file @ d3180f02
dockerfiles/commul-customization/webpage/images/ercc-logo.png

161 KiB | W: 0px | H: 0px

dockerfiles/commul-customization/webpage/images/ercc-logo.png

68.3 KiB | W: 0px | H: 0px

dockerfiles/commul-customization/webpage/images/ercc-logo.png
dockerfiles/commul-customization/webpage/images/ercc-logo.png
dockerfiles/commul-customization/webpage/images/ercc-logo.png
dockerfiles/commul-customization/webpage/images/ercc-logo.png
  • 2-up
  • Swipe
  • Onion skin
dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png
View replaced file @ e962ee7d View file @ d3180f02
dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png

642 B | W: 0px | H: 0px

dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png

453 B | W: 0px | H: 0px

dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png
dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png
dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png
dockerfiles/commul-customization/webpage/images/favicon_eurac-research.png
  • 2-up
  • Swipe
  • Onion skin
dockerfiles/commul-customization/webpage/index.html
View file @ d3180f02
......@@ -28,7 +28,7 @@
<div id="heading">
<h1>Welcome to the Eurac Research Clarin Centre!</h1>
</div>
<img src="images/clarin_ill_1100x375_2.jpg" style="max-width:100%" width="1200" height="410" />
<img src="images/clarin_ill_1100x375_2.png" style="max-width:100%" width="1200" height="409" />
</div>
<div id="maintext">
<p>The Eurac Research CLARIN Centre (ERCC) is a dedicated repository for language data. It is hosted by the Institute for Applied Linguistics (IAL) at Eurac Research, a private research centre based in Bolzano, South Tyrol. The Centre is part of the Europe-wide CLARIN infrastructure, which means that it follows well-defined international standards for (meta)data and procedures and is well-embedded in the wider European Linguistics infrastructure. The repository hosts data collected at the IAL, but is also open for data deposits from external collaborators.</p>
......
kubernetes/dspace-deployment.yaml
View file @ d3180f02
......@@ -50,7 +50,7 @@ spec:
secretKeyRef:
key: mail.pass
name: dspace-secrets
image: gitlab.inf.unibz.it:4567/commul/docker/clarin-dspace/production/dspace:1.2.2
image: gitlab.inf.unibz.it:4567/commul/docker/clarin-dspace/production/dspace:1.3
name: clarin-dspace
imagePullPolicy: Always
ports:
......
kubernetes/dspace-ingress.yaml
View file @ d3180f02
......@@ -2,20 +2,26 @@ apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/add-base-url: "true"
ingress.kubernetes.io/ssl-passthrough: "true"
# OAI-PMH uses HTTP transport, ie. we can't redirect *all* traffic to https.
# However, we need to make sure that 'our internal' nginx redirects http:// -> https://
# (except for /repository/oai).
ingress.kubernetes.io/ssl-redirect: "false"
# But just to be sure, we are /also/ redirecting here.
ingress.kubernetes.io/configuration-snippet: |
if ($scheme != "https") {
set $rewrite_https 1;
}
if ($request_uri ~ "^/repository/oai") {
set $rewrite_https 0;
}
if ($rewrite_https = 1) {
return 301 https://$server_name$request_uri;
}
ingress.kubernetes.io/secure-backends: "true"
name: dspace-ingress
namespace: dspace
spec:
rules:
- host: clarin.eurac.edu
http:
paths:
- backend:
serviceName: nginx
servicePort: 80
path: /repository/oai
- host: clarin.eurac.edu
http:
paths:
......@@ -26,3 +32,4 @@ spec:
tls:
- hosts:
- clarin.eurac.edu
secretName: host-tls-cert
kubernetes/nginx-deployment.yaml
View file @ d3180f02
......@@ -20,7 +20,7 @@ spec:
io.kompose.service: nginx
spec:
containers:
- image: gitlab.inf.unibz.it:4567/commul/docker/clarin-dspace/production/nginx:1.2.2
- image: gitlab.inf.unibz.it:4567/commul/docker/clarin-dspace/production/nginx:1.3
imagePullPolicy: Always
name: clarin-nginx
ports:
......