Welcome to the Cyclos 4 PRO Documentation for version 4.16.8. For other versions, see https://www.cyclos.org/documentation.

There are also some other important documentation resources that are not part of this manual:

1. Cyclos setup

This is the installation manual for Cyclos 4 PRO. Cyclos is server side software. End users (customers) will access Cyclos using a web browser or mobile phone.

If you have any problems when installing Cyclos using this manual, you can ask for help on our forum.

Cyclos can be installed in a Tomcat server or by using a Docker container. If you want to have a quick preview of Cyclos it is easier to use Docker (especially on Linux).

1.1. Installation

1.1.1. Installation with Tomcat

This method will deploy Cyclos as a web application of an existing Apache Tomcat server.

System requirements
  • Operating system: Linux (x86_64 or arm64), Windows (64) or Mac;

  • At least 2 GB memory available for the JVM;

  • Java Runtime Environment (JRE), Java 11 is required;

  • Web server: Apache Tomcat 9.0. Tomcat 10+ implements Jakarta EE 10, which is not compatible with Cyclos 4.16.8;

  • Database server: PostgreSQL 12 or higher;

  • Cyclos installation package cyclos-4.16.8.zip;

Install Java

You can check if you have Java at least with version 11 installed by opening a command prompt and typing this:

java --version

If you don’t have Java, or the version is below 11, proceed with the steps below:

Linux (Ubuntu)

Run the following command:

sudo apt install openjdk-{java-version}-jre

Windows

Install the PostgreSQL database

Linux (Ubuntu)

sudo apt install postgresql postgis

Then, test your installation:

sudo -u postgres psql

If you see postgres=# you are in the PostgreSQL command line, and you can follow the instructions below.

Windows

  • Download the latest version of PostgreSQL and PostGIS;

  • Install both PostgreSQL and PostGIS by following the installer steps (use the default options);

  • Make sure the bin directory is included in the system variables, so that you can run psql directly from the command line:

    • Go to: Start > Control Panel > System and Security > System > Advanced system settings > Environment Variables…;

    • Then go to the system variable with the name "Path" and add the bin directory of the PostgreSQL installation directory as a value, such as C:\Program Files\PostgreSQL\14\bin. Don’t forget to separate the values with a semicolon.

  • Go to the Windows command line and type the command (you will be asked for the password you specified when installing PostgreSQL):

psql -U postgres

If you see postgres=# you are in the PostgreSQL command line, and you can follow the instructions below.

Create the Cyclos database

After configuring the PostgreSQL server, you should run the following commands in psql (same command as listed below to test the PostgreSQL connection):

create user cyclos with encrypted password 'cyclos-password';
create database cyclos4 encoding 'utf-8' template template0 owner cyclos;
\c cyclos4
create extension cube;
create extension earthdistance;
create extension postgis;
create extension unaccent;
create extension pgcrypto;

Then exit the psql command by typing \q and pressing enter.

Install Tomcat web server
  • Download Tomcat 9 (cannot be 10+ because of the Jakarta namespace changes) from https://tomcat.apache.org/;

  • Extract the zipped file into a folder <tomcat home>;

  • Start tomcat: <tomcat home>/bin/startup.bat (Windows) or <tomcat home>/bin/startup.sh (Linux). You might have to give the execution permissions to the file;

  • Open a browser and go to http://localhost:8080/ to check for installation;

  • The default JVM memory heap size of Tomcat is very low, we recommend increasing it (see adjustments).

Install Cyclos

Make sure Tomcat is working on port 8080 of the local machine. Also make sure the user running Tomcat has permissions to write in the webapps directory. Then

  • Visit our license server at https://license.cyclos.org/;

  • Register yourself. With the registration, you can run free licenses, which allows running systems with up to 300 users;

  • Download the latest Cyclos version;

  • Unzip the cyclos-version.zip into a temporary directory;

  • Browse to the temporary directory and copy the directory web (including its contents) into the webapps directory (<tomcat_home>/webapps) of the Tomcat installation;

  • Rename this web directory to the name that you will want to use at the URL, in this example we will use instance_name. This name will define how users will access Cyclos. For example, if you run the tomcat server on https://www.domain.com. the URL would be http://www.domain.com/instance_name. Cyclos has some reserved words that cannot be used for the instance name:

cyclos.gwt
fonts
js
pay
consent
unsubscribe
voucher
classic
ui
.well-known
robots.txt
sitemap.xml
sitemap-index.xml
sitemap.xstl
activate-access-client
external-redirect-callback
identity
run
content
web-rpc
java-rpc
api
sms
push-notifications
global
redirect
mobile-redirect
  • Of course, it is also possible (and recommended) to run Cyclos directly under the domain name. This can be done by renaming the web directory to ROOT. You should first remove the existing ROOT directory;

  • In the folder <tomcat_home>/webapps/<instance_name>/WEB-INF/classes, you’ll find the file cyclos-release.properties. Copy this file, giving it the name cyclos.properties. The original name is not shipped, so in future installations you can just override the entire folder, and your customizations won’t be lost;

  • In the cyclos.properties file, you can set the database configuration. Here you have to specify the username and password, by default it is set as cyclos4 as database name and cyclos as username and password. For production, it is recommended to change the password:

cyclos.datasource.provider = hikari
cyclos.datasource.dataSourceClassName = org.postgresql.ds.PGSimpleDataSource
cyclos.datasource.dataSource.portNumber = 5432
cyclos.datasource.dataSource.serverName = localhost
cyclos.datasource.dataSource.databaseName = cyclos4
cyclos.datasource.dataSource.user = cyclos
cyclos.datasource.dataSource.password = cyclos
  • Notes:

    • Some systems do not resolve localhost and the default PostgreSQL port directly. In case of database connectivity problems, try replacing it with the IP address of your system;

    • Windows might not see line breaks in the property file, if this is the case we advise you to download a more advanced text editor such as Notepad++;

    • On Windows, in case of problems, you can change the cyclos.tempDir setting in cyclos.properties. Point it to a temp` directory inside the WEB-INF directory in Cyclos. E.g. cyclos.tempDir = C:\Program Files\Tomcat9\webapps\instance_name\WEB-INF\temp. In some cases, even forward slashes need to be used.

  • Afterward, specially in Linux systems, make sure that all files inside <tomcat_home>/webapps/<instance_name> can be read / written by the system user that will run the Tomcat service.

Start Cyclos
  • (Re)start Tomcat:

    • Stop with <tomcat_home>/bin/stop.bat (Windows) or <tomcat_home>/bin/stop.sh (Linux);

    • Start with <tomcat_home>/bin/startup.bat (Windows) or <tomcat_home>/bin/startup.sh (Linux);

    • Windows: you can use Tomcat monitor (available after tomcat installation).

  • When Tomcat is started and Cyclos initialized, open a web browser to http://localhost:8080/instance_name. Be aware, starting up Cyclos for the first time might take quite some time, because the database needs to be initialized. On a slow computer, this could take a few minutes!;

  • Upon the first start of Cyclos you will be asked to fill in the username and password you used for registration in the license server. You will also need to provide the information of a global administrator;

  • After submitting the correct information, the initialization process will finish, and you will be automatically logged-in as the created global administrator;

  • You will be presented with the network wizard to create the default network.

Upgrading to a newer version

Before upgrading, please, carefully follow these steps.

Then, to upgrade:

  • Download the latest cyclos-<version>.zip file from the license server;

  • Unzip the file to a temporary directory;

  • Stop the Tomcat server;

  • Remove all files in <tomcat_home>/webapps/<instance_name>/WEB-INF/lib;

  • Copy all files and directories from the temporary directory's web folder back to <tomcat_home>/webapps/<instance_name>, overwriting all existing files;

  • Start the Tomcat server.

Problem-solving
  • Often, problems can be easily detected by looking at the log files:

    • The Tomcat’s <tomcat-home/logs/catalina.out file;

    • Cyclos' own log files, as configured in cyclos.properties. The Cyclos log shows all relevant information about the services and tasks that run in Cyclos.

  • If the logs can’t help you to pin down the problem, you can search the Cyclos forum (installation issues) if somebody encountered a similar problem;

  • If you can’t find an answer, post a new question in that same forum topic;

  • In case you locked yourself out of the system, see the section Reset admin password directly on database.

1.1.2. Installation with Docker

There is a Docker image for Cyclos, and the installation via Docker is very easy, and can be accomplished with a few steps. It can also be used for production with no drawbacks when compared to the Tomcat installation.

For details on how to install Cyclos via Docker image, visit the Cyclos repository on Docker hub. It contains detailed information on installation and maintenance.

For details on how to install docker, please visit https://docs.docker.com/engine/install/.

1.1.3. Installation with Kubernetes (EKS)

This method will deploy Cyclos with Kubernetes using Amazon Elastic Kubernetes Service (EKS). Kubernetes is the industry standard system for cluster orchestration, but the setup process is provider-dependent. Many of the following concepts and steps can be adjusted for other providers.

Please, note that there are many steps to follow. Please, follow each one carefully!

Requirements

You will need the following CLI tools. Please, refer to their respective websites for installation instructions:

Create the required IAM roles

You need a separated IAM role for the cluster to use (Cluster service role). You’ll also need one for the node group. For this, go to the Amazon IAM console, and go to 'Roles'.

Create the cluster role with the following:

  • Trusted entity type: AWS service

  • Use cases for other AWS services: EKS

  • Check EKS - Cluster

Choose 'Next' and 'Next' again. In 'Name, review and create' set the name cyclos-eks, and a description, then click on 'Create role'.

Then create a new role for the node group with the following:

  • Trusted entity type: AWS service

  • Use case: EC2

Choose 'Next'. In the 'Add permissions' page, copy and paste each of these permissions, checking the checkbox next to them for each one:

  • AmazonEKSWorkerNodePolicy

  • AmazonEC2ContainerRegistryReadOnly

  • AmazonEKS_CNI_Policy

Choose 'Next'. In 'Name, review and create' set the name cyclos-eks-nodes, and a description, then click on 'Create role'.

Create separated VPC subnets

It is required to have at least 2 subnets in your VPC, in different availability zones, which auto-assign IPv4 addresses. For this, in the Amazon VPC console, click on 'Subnets', then create a new one. Choose your VPC, set a name, choose an availability zone, type in a valid IPv4 CIDR block (also for IPv6 if desired), then add the following tag (without it the load balancer won’t work):

  • Key: kubernetes.io/role/elb

  • Value: 1

Then select the subnet you just created, click 'Actions' and select 'Edit subnet settings'. Check the 'Enable auto-assign public IPv4 address' and save it.

Then repeat all these steps for creating another subnet, just select another availability zone.

Configure the AWS CLI

If you already use the AWS CLI, skip this step.

In order to use the AWS CLI, you will need a 'Security credential'. In the Amazon console, in the top bar at the right, in your logged username, select 'Security credentials'. Scroll down to 'Access keys' and create a new one if you don’t have it yet. 'Choose Command Line Interface (CLI)' and click 'Next'. Then 'Create access key'.

It will show you both 'Access key' and 'Secret access key'. You’ll need both.

Now open a terminal console and type in:

aws configure

Paste your 'Access key', then the 'Secret access key', then select the region you use by default and finally, as the output format, we suggest json.

Create the cluster

Create a cluster in the Amazon EKS console. In 'Clusters', choose 'Add cluster' > 'Create'. Select the following:

  • Name: cyclos (if you change it, you’ll need to adjust many of the other steps as well)

  • Kubernetes version: Choose either the default or the latest one

  • Cluster service role: cyclos-eks (the role previously created)

Click 'Next'.

  • VPC: Choose your VPC

  • Subnets: Choose the 2 subnets you’ve created previously

  • Security groups: Choose a security group that opens the ports 80 and 443 for public access. If you don’t have one, create one.

  • Cluster endpoint access: Public and private

Click 'Next'. You may turn on logging if desired, then click 'Next' again.

You will be presented with the add-ons. Leave the defaults and click 'Next'. Leave the defaults and click 'Next' again.

Finally, click on 'Create'. It will take a few minutes until the cluster is actually created.

Connect to your cluster

You’ll need to run the following command to update the kubectl configuration to point to your cluster. For this, run the following:

aws eks update-kubeconfig --name cyclos

Make sure the cluster has finished creating. If not, it will fail with the message Cluster status is CREATING.

Then, test the connection to your cluster by typing in.

kubectl cluster-info

If you try it with an Amazon user which is not the one that created the cluster, an error will be presented. To fix it, grant permission to the current user with the following command, which must be executed by the cluster creator user (adjust the <account-id>, <current-user> and <cluster-creator-user> variables):

eksctl create iamidentitymapping \
    --cluster cyclos \
    --arn arn:aws:iam::<account-id>:user/<current-user> \
    --group system:masters \
    --no-duplicate-arns \
    --username <cluster-creator-user>
Create a node group

In your cluster details in Amazon EKS console, choose the 'Compute' tab and click 'Add node group'. Select the following:

  • Name: cyclos-nodes

  • Node IAM role: cyclos-eks-nodes

Click 'Next'. Choose an AMI type (both x86_64 and ARM_64 architectures are supported). Choose the desired capacity, instance type (we recommend a minimum 4 GB of RAM and 2 CPUs, but depending on the system scale a better hardware is desirable) and disk size (recommended 20 GB). Also choose the desired scaling configuration (choose at least 2 nodes to ensure high availability). Click 'Next'.

Then select the 2 subnets you have previously created (which automatically assign public IPv4 addresses). Click on 'Create'. It will take several seconds to create the node group.

To check that the nodes were created, run:

kubectl get nodes
Deploy the database service

This guide assumes that you have a PostgreSQL 12+ database running. For production, we recommend using an 'Amazon RDS' with Engine type 'Aurora (PostgreSQL compatible)' database. In the database details in the Amazon RDS console, you have the 'Endpoint', which is the hostname that resolves to the database server.

Create locally a file named postgresql.yaml with the following content, replacing <database-endpoint> with the proper hostname:

kind: "Service"
apiVersion: "v1"
metadata:
  name: "postgresql"
spec:
  type: ExternalName
  externalName: <database-endpoint>

Then create the service with:

kubectl apply -f postgresql.yaml
Deploy the Cyclos service

To deploy Cyclos, create locally a file named cyclos.yaml with the following content, replacing the following variables:

  • <replicas>: The desired number of Cyclos cluster instances;

  • <db-name>: The database name in your PostgreSQL server;

  • <db-user>: The database user in your PostgreSQL server;

  • <db-password>: The database password in your PostgreSQL server;

  • <cyclos-version>: The desired Cyclos version. At the time of writing of this document, the most recent version was 4.16.8.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cyclos
spec:
  replicas: <replicas>
  selector:
    matchLabels:
      app.kubernetes.io/name: cyclos
  template:
    metadata:
      labels:
        app.kubernetes.io/name: cyclos
    spec:
      containers:
      - name: cyclos
        image: cyclos/cyclos:<cyclos-version>
        ports:
        - containerPort: 8080
          name: cyclos-web-port
        env:
        - name: JAVA_OPTS
          value: "-Xmx2G"
        - name: CLUSTER_K8S_DNS
          value: cyclos-dns
        - name: DB_HOST
          value: postgresql
        - name: DB_NAME
          value: <db-name>
        - name: DB_USER
          value: <db-user>
        - name: DB_PASSWORD
          value: <db-password>

---

apiVersion: v1
kind: Service
metadata:
  name: cyclos-dns
spec:
  type: ClusterIP
  clusterIP: None
  selector:
    app.kubernetes.io/name: cyclos

Then create the service with:

kubectl apply -f cyclos.yaml

To verify the pods are using the correct image, run the following and verify the 'Image' field:

kubectl describe pods

Please, note that the first startup will take several seconds, specially if the database is being populated for the first time. You can also check the logs of each Cyclos pod with (replacing the <deployment> and <pod> variables with the ones returned by the previous command):

kubectl logs -f cyclos-<deployment>-<pod>

This will block your terminal and update with new logs. Press Control+C to exit the log viewing and return to the terminal prompt.

Publishing Cyclos

The Cyclos service needs to be publicly accessible, with the following requirements:

  • The service needs to be accessible using a friendly URL, such as https://account.my-domain.com;

  • An SSL certificate is required in order to use the secure (HTTPS) protocol;

  • Incoming requests using the non-secure (HTTP) protocol will be redirected, switching to HTTPS;

  • Incoming requests need to be load-balanced into any of the Cyclos deployment replicas (pods).

To do so, an Amazon 'Application Load Balancer' (ALB) is required. The default, 'Classic load balancer', doesn’t support redirecting requests. Setting it up require several steps. Please, follow each of them carefully. Also, this documentation assumes your domain is managed by Amazon.

Creating an SSL certificate

In the Amazon Certificate Manager console, click in the 'Request' button. Choose 'Request a public certificate', and click 'Next'. Then you need to type in the target domain name. You can leave the rest of the options in their default values. Then press 'Request'.

The certificate is now in the pending validation status. To fix this, you need to go to the certificate details and click on the 'Create records in Route 53' button. It will validate the certificate. Take note of the certificate 'ARN'.

Setting up the load balancer roles

First, you will need to associate the 'OpenID Connect provider' role for the cluster:

eksctl utils associate-iam-oidc-provider --cluster cyclos --approve

Then register an IAM policy. You need to download the latest release of the iam_policy.json file to a local folder. To get which is the latest version, visit the releases page, taking note of the latest release (should be something like v2.4.6). Then download the file, replacing the <version> variable:

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/<version>/docs/install/iam_policy.json

Now create the IAM policy:

aws iam create-policy \
  --policy-name AWSLoadBalancerControllerIAMPolicy \
  --policy-document file://iam_policy.json

Now you need to create an IAM role for the load balance controller. Replace the <account-id> variable with your AWS account id:

eksctl create iamserviceaccount \
  --cluster=cyclos \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::<account-id>:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve
Install the load balancer controller

Now you need to install the load balancer controller in your cluster, using Helm. First add the EKS repository:

helm repo add eks https://aws.github.io/eks-charts

Now update the local repositories:

helm repo update

Finally, install the AWS load balancer controller:

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=cyclos \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

Now you need to create the Kubernetes' Ingress controller. It will be responsible for:

  • Receiving incoming HTTP requests;

  • Ensure the HTTPS protocol is used (will redirect from plain HTTP to HTTPS);

  • Validate the SSL certificate;

  • Forward requests to one of the Cyclos pods (load-balancing them).

To create this service, create a local file named ingress-controller.yaml with the following content (remember to update the <certificate-arn> variable with the ARN of the SSL certificate you’ve created before):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-srv
  annotations:
    # Ingress Core Settings
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    ## SSL Settings
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: <certificate-arn>
    # SSL Redirect Setting
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: ssl-redirect
                port:
                  name: use-annotation
          - path: /
            pathType: Prefix
            backend:
              service:
                name: cyclos-server
                port:
                  number: 80
---
apiVersion: v1
kind: Service
metadata:
  name: cyclos-server
spec:
  selector:
    app.kubernetes.io/name: cyclos
  ports:
    - name: http
      port: 80
      targetPort: 8080
  type: NodePort

Now actually create the service in Kubernetes:

kubectl apply -f ingress-controller.yaml

If there was an error, delete the IAM service account with eksctl delete iamserviceaccount --cluster=cyclos --namespace=kube-system --name=aws-load-balancer-controller, uninstall the service with helm uninstall aws-load-balancer-controller -n kube-system and retry the previous steps.

To verify that the controller is running, execute the following:

kubectl get deployment -n kube-system aws-load-balancer-controller

Also, check the created load balancer identifier by running:

kubectl get ingress

You will need this load balancer identifier for the next step.

Update the load balancer DNS

In the Amazon Route 53 console, select 'Hosted zones'. Select your zone. Click on 'Create record'. In the record name, type in the subdomain. The 'Record type' must be 'A'. Then check the 'Alias' toggle. Then set 'Route traffic to' as 'Alias to Application and Classic Load Balancer'. Select your region below. Then select the load balancer that was created as noted in the previous step (it may be prefixed with dualstack.). Click on 'Create records' and you’re done.

Finally, open a web browser pointing to your domain, and you should have access to your instance.

Upgrading to a newer version

Before upgrading, please, carefully follow these steps.

Important! Cyclos doesn’t support rolling updates because it needs to upgrade the database schema.

So, first you need to stop all cluster instances:

kubectl scale --replicas=0 deployments/cyclos

Then you need to update the Cyclos deployment to use the new image:

kubectl set image deployments/cyclos cyclos=cyclos/cyclos:<new-version>

Finally, scale the deployment back to the desired number of replicas:

kubectl scale --replicas=<replicas> deployments/cyclos

1.2. Upgrading to a newer version

IMPORTANT! Before upgrading, always:

  • Carefully review the release notes in the license server, specially with milestone upgrades, such as from 4.10 to 4.11. Sometimes there are new system requirements (newer Java / PostgreSQL versions, for example), or manual actions that could make the upgrade fail if not applied first (such as creating a new extension in PostgreSQL);

  • Make a backup (dump) of the database. New Cyclos versions generally contain new functionality that requires the database to be modified. With milestone upgrades these changes can be major, and, in case of failure, you should always have a backup to be able to restore;

  • First upgrade a test environment. This is very important. Specially for milestone releases, never upgrade the production instance without testing it first! See Setup a test environment for more details;

  • Specially with milestone upgrades, test all your scripts and API calls;

  • Milestone versions can include new settings in cyclos-release.properties. It might be interesting to study the new file to see if a new setting can be useful for your installation;

  • Shut down all members when running in a cluster to avoid problems due to version mismatches. From Cyclos 4.16.8 onwards an error is generated if the versions differ

If everything went well in the test environment, then upgrade the live environment.

Note for upgrades from versions prior to 4.15: In Cyclos 4.15 a new PostgreSQL extension is required: pgcrypto. It requires being created as superuser. If the database user configured in cyclos.properties is not a superuser, please connect to the Cyclos database and run the following commands as a superuser:

create extension pgcrypto;
drop index if exists ix_background_task_executions;
create unique index ix_background_task_executions on background_task_executions (class_name, digest(context, 'sha512'));

1.3. Adjustments (optional)

There are many additional steps for configuring Cyclos, specially for production, as well as tuning it for large systems.

1.3.1. Adjust Tomcat/Java memory

The default memory heap size of Tomcat is very low. You can augment this in the following way:

Windows

In the bin directory of Tomcat create (if it doesn't exist) a file called setenv.bat, edit this file and add the following line:

set JAVA_OPTS=-Xmx2g

Linux

In the bin directory of Tomcat create (if it doesn't exist) a file called setenv.sh, edit this file and add the following line:

JAVA_OPTS="-Xmx2g"

1.3.2. Handling out of memory errors

When an OutOfMemoryError (OOME) is unhandled, it is generally advisable to kill the JVM. Although any operation can, theoretically, generate `OOME’s, one particular one is prone to it: generating PDFs. Cyclos uses the excellent openhtmltopdf library, which transforms HTML documents to PDFs. However, as most HTML handling applications (such as browsers), the RAM requirement is high, depending on the size of the HTML document being processed.

When generating PDFs, Cyclos attempts to catch `OOME’s. However, if some other Tomcat thread gets out of memory in the same time, the entire server will become unresponsive, and there will be no additional choice than restarting the server.

The recommended approach is to use pass a JVM property (in a similar fashion to memory adjustments) to kill the JVM process in such cases. It is then the responsibility of an external service monitoring tool to restart the Tomcat server. The parameter to pass is -XX:OnOutOfMemoryError="kill -9 %p".

In most systems, Cyclos is either:

  • Executed via Docker. Starting with Cyclos 4.16.2, this parameter is included in the default in the Docker image. For previous versions, just set the environment variable JAVA_OPTS="-XX:OnOutOfMemoryError=\"kill -9 %p\"". Just remember to pass the --init --restart=unless-stopped arguments when starting the container. Without the --init, the java process will have pid 1, which is not killable in Linux;

  • Executed in a Linux server using Systemd. In this case, you should set Restart=on-failure and pass the JAVA_OPTS environment variable. As Systemd replaces %p by its service name, you should escape it with %%p. Here’s an example:

[Unit]
Description=Apache Tomcat for Cyclos
After=syslog.target network.target

[Service]
User=tomcat
Group=tomcat
Type=forking
Environment=CATALINA_PID=/opt/tomcat/cyclos.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/cyclos
Environment=JAVA_OPTS="-Djava.awt.headless=true -XX:OnOutOfMemoryError=\"kill -9 %%p\""
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh
Restart=on-failure

[Install]
WantedBy=multi-user.target

In both cases, if the Tomcat server has an unhandled OOME, it will be restarted instead of becoming unresponsive.

1.3.3. Enable SSL/HTTPS

Enabling SSL is crucial on live systems, as it protects sensitive information, like passwords, to be sent plain over the Internet, making it readable by eavesdroppers.

Generally it is advised to use a proxy server, like Apache or Nginx, that handles HTTPS and then redirect the request to Tomcat. See Enable SSL on Apache for more details.

Otherwise, if the Tomcat server is directly accessible from the Internet, to enable SSL / HTTPS you first have to enable (uncomment) the https connector in the file <tomcat_home>/conf/server.xml

<Connector port="443" maxHttpHeaderSize="8192"
    maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
    enableLookups="false" disableUploadTimeout="true"
    acceptCount="100" scheme="https" secure="true"
    clientAuth="false" sslProtocol="TLS" />

Generate a key with the keytool utility that comes bundled with Java:

keytool -genkey -alias tomcat -keyalg RSA -keystore /path/to/my/keystore

After executing this command, you will first be prompted for the keystore password. Passwords are case-sensitive. You will also need to specify this password in the server.xml configuration file, as described later.

Next, you will be prompted for general information about this certificate, such as company, contact name, and so on. This information will be displayed to users who attempt to access a secure page in your application, so make sure that the information provided here matches what they will expect.

Finally, you will be prompted for the key password, which is the password specifically for this certificate (as opposed to any other Certificates stored in the same keystore file). You MUST use the same password here as was used for the keystore password itself, or Tomcat will have problems loading it. Currently, the keytool prompt will tell you that pressing the ENTER key does this for you automatically.

If everything was successful, you now have a keystore file with a certificate that can be used by your server.

1.3.4. Clustering

Clustering is useful both for scaling (serving more requests) and for high availability (if a server crashes, the service continues to run). Please note that when upgrading Cyclos to a newer version the entire cluster must be shut down which will cause some downtime.

There's no need to configure a Tomcat-level cluster, because it is only used to replicate HTTP sessions. Cyclos, however, doesn't use Tomcat sessions, but handles them internally. This way, there is no special Tomcat configuration to support a Cyclos cluster.

The Cyclos application, however, needs some small configurations to enable clustering. Cyclos uses Hazelcast to synchronize shared state (such as caches) between cluster hosts. To enable clustering, find in cyclos.properties the line containing cyclos.clusterHandler, and set it to hazelcast.

Hazelcast is able to find the cluster nodes with many different join strategies: multicast (for local network), TCP/IP (when the list of nodes is known beforehand), Kubernetes (using a DNS service), Amazon Web Services (by matching nodes via tags) and Google Cloud Platform (by matching nodes via labels). Starting with Cyclos 4.16, it is possible to both enable clustering and configuring the join strategy by setting one of these environment variables:

  • CLUSTER_MULTICAST: Enables clustering with multicast join. The value should be in the format group:port, for example, 224.2.2.3:54327;

  • CLUSTER_TCPIP: Enables clustering with TCP/IP join. The value should be a list of comma-separated members, either host or host:port. For example: 10.0.0.1,10.0.0.2:5108;

  • CLUSTER_K8S_DNS: Enables joining in Kubernetes using a DNS service. The value should be the DNS service name, for example, cyclos-dns;

  • CLUSTER_AWS_TAG: Enables Amazon Web Services (AWS) join using a tag key and value for finding nodes that should join the cluster. The value should be in the format tagKey=tagValue. Make sure to apply the given tag to the EC2 nodes metadata;

  • CLUSTER_GCP_LABEL: Enables Google Cloud Platform (GCP) join using a label key and value for finding nodes that should join the cluster. The value should be in the format labelKey=labelValue. Make sure to apply the given label to the node’s metadata.

Another way to configure the join strategy, as well as fine-tuning other parameters, is by configuring the WEB-INF/classes/hazelcast.xml file. Check the Hazelcast documentation for more details. When one of the previously mentioned environment variables are found, any <join> tags in hazelcast.xml are ignored, and the environment variable is used instead.

To set up high-availability at database (PostgreSQL) level, it is recommended to either use a managed PostgreSQL service from a cloud provider or deploy a PostgreSQL cluster with CloudNativePG.

1.3.5. Use Apache with mod_jk

You can use Apache as a front-end / load balancer for Tomcat. This is very useful when you have several domains configured on the server. There are several documentations and examples available on the Internet. In our example, we will use the module mod_jk.

sudo apt install apache2 libapache2-mod-jk

The configuration is done in the /etc/libapache2-mod-jk/workers.properties file. By default, this is configured to use the AJP port 8009, this is the default AJP port for Tomcat, if you are using a different port you need to configure it here.

On Tomcat, we need to enable the AJP connector. Edit the file <tomcat_home>/conf/server.xml and uncomment the AJP connector:

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" secretRequired="false" />

Now on Apache we need to configure the virtualhost (which on Ubuntu can be found here in /etc/apache2/sites-enabled/) to use the AJP connector. On the virtualhost of your domain, add the following lines:

<IfModule mod_jk.c>
    JkMount /* ajp13_worker
    JkMount / ajp13_worker
</IfModule>

This example uses the cyclos as ROOT application on Tomcat. If you want to use something like http://www.yourdomain.com/instance_name we need to deploy cyclos on the webapps/instance_name` directory and configure apache like this:

<IfModule mod_jk.c>
    JkMount /instance_name/* ajp13_worker
    JkMount /instance_name ajp13_worker
</IfModule>

Now restart both Apache and Tomcat and check if it works.

1.3.6. Enable SSL on Apache

Enabling SSL is crucial for live systems, as it protects sensitive information, like passwords, to be sent plain over the Internet, making it readable by eavesdroppers.

We recommend using Let’s Encrypt to set up a certificate. It is free and has only a few steps for setup.

1.3.7. Setup a proxy / load balancer

The easiest configuration for a load balancer is Apache connecting to Tomcat using the AJP protocol. In this case, the original request is forwarded to Tomcat as is, keeping the original client IP address and URL.

However, in most other cases, the load balancer works as a proxy, sending a new HTTP to Tomcat and forwarding the response to the client. Examples of such proxies include Apache with mod_proxy, Nginx, haproxy and all cloud provider load balancers.

In either way, generally the proxy will have the server certificate and will terminate the SSL connection with the client. Then an internal (non-HTTPS) request is performed from the proxy to Tomcat.

That means the client IP address received by Cyclos, as well as the request URL, are different from the original request performed by the client. As Cyclos uses the client IP for logging and blocking in case of abuse, this would lead Cyclos to block the proxy, preventing any further request.

However, the proxy will add some extra request headers, with information about the original request. Cyclos then needs to be configured to read both the IP address and which was the connection protocol used by the original request (HTTP or HTTPS) from those headers, instead of directly from the incoming HTTP request. For this, the following settings in cyclos.properties are needed:

  • cyclos.header.remoteAddress: Specifies the name of the header which contains the original client’s IP address. The name of this header is usually X-Forwarded-For. The value associated to this header will be a comma-separated list of IPs;

  • cyclos.header.remoteAddress.index: Specifies the position in the list to read the client’s IP from. The default value is 0. To avoid a malicious user perpetrating an IP spoofing attack, we strongly recommend using a negative value indicating an offset from the end of the list. Thus, -1 means the last IP, -2 the previous one, and so on;

  • cyclos.header.protocol: Specifies the protocol name (http or https) used on the original request. The name of this header is usually X-Forwarded-Proto.

The following cases are handled by Cyclos to match a specific network / configuration from the request URL:

  • A request to Tomcat using the root URL specified in a parent configuration (normally the network default). For example, if the network default configuration’s root URL is http://cyclos-net.com, any requests to http://cyclos-net.com/* will match that configuration;

  • A request to Tomcat using the root URL specified in a parent configuration plus a specific path of an inherited configuration. Following the previous example, if the child configuration has a custom path of config, any requests to http://cyclos-net.com/config/* will match that configuration;

  • If no custom URL is matched, requests having the first subpath (after the web application context path) equals the network internal name. For example, if Cyclos is deployed in a Tomcat under the context path cyclos, and it has a network called main, any requests to http://localhost:8080/instance_name/main/* will match this network;

  • Same as previous, but with a specific configuration path. Following the previous example, if that network has a configuration with path config, requests to http://localhost:8080/instance_name/main/config/* will match this configuration;

  • Also, the name global is reserved as a network internal name. For example, requests to http://localhost:8080/instance_name/global/* will be considered in global mode;

  • Still, if no custom URL is matched, if no network internal name is given and there is a default network, the network internal name can be omitted. For example, requests to http://localhost:8080/instance_name/* will be considered in the default network.

When Tomcat is behind a proxy, it will never receive requests using the original public URL. Hence, only matching by network (and optionally, configuration paths) will be used.

Still, networks should correctly set the root url in their configurations because they are used to generating full URLs at the server side, for example, when sending links in e-mails or resolving image URLs in rich texts.

Following are common cases that could be configured for a proxy, all assuming Cyclos is deployed to a Tomcat accessible by the proxy via http://tomcat:8080/instance_name:

1.3.8. Reserved path names

There are reserved paths in Cyclos that cannot be used as paths in proxies. For example, a proxy could handle requests to https://www.my-project.com/app and redirect them to http://tomcat:8080/instance_name. In this case, the /app path part is public, used by clients, but never visible on Tomcat.

To handle such cases, the list of reserved paths is used to generate the correct URIs for scripts and stylesheets, and having any of the reserved paths in the proxy would prevent the URI generation from working correctly.

The list of reserved paths is:

cyclos.gwt
fonts
js
pay
consent
unsubscribe
voucher
classic
ui
.well-known
robots.txt
sitemap.xml
sitemap-index.xml
sitemap.xstl
activate-access-client
external-redirect-callback
identity
run
content
web-rpc
java-rpc
api
sms
push-notifications
global
redirect
mobile-redirect

1.3.9. Enabling Google Maps

Cyclos supports displaying maps using Google Maps. This has to be enabled in the Cyclos configuration. Cyclos also uses geocoding to map the user-informed address fields to a position (latitude/longitude).

Google Maps requires an API key. For details on the free daily quota for map views and geocode requests, see this page.

There are actually 2 API keys that can be set in the Cyclos configuration: The server-side API key and the browser API-key.

Each one needs to be generated in the API Manager.

  • Enabling the APIs: on the "Library" menu, search for the following APIs and enable them: "Maps JavaScript API", "Maps Static API" and "Geocoding API";

  • Creating the API keys: on the "Credentials" menu, choose "Create credentials", then choose "API key". Choose "Server key" and specify a name for it. Save and then create a new one, this time as "Browser key".

Once the API keys are ready, they can be copied / pasted into the Cyclos configuration, on the corresponding "Google maps server API" and "Google maps browser API" fields.

The API Manager allows monitoring of requests performed by each API.

On the main Cyclos web application, addresses are geolocated on the client-side, before saving it. In such cases, the browser API key is used. However, sometimes addresses can be saved without being geolocated, either by third-party software or by importing new users into Cyclos. In such cases, a background task will attempt to geolocate them (might take up to 24 hours for this) using the server API key.

1.3.10. reCAPTCHA

Starting with Cyclos 4.15, it has been added the support for reCAPTCHA v2, which displays the "I am not a robot" checkbox. As of 2021, it offers a free usage for up to 1 million requests per month. As CAPTCHAS are used in Cyclos only on registration and on 'forgot password' requests, it should be enough for most systems.

To enable it, first you need to register a new site in reCAPTCHA. Select the v2 on reCAPTCHA type. Then add your domain, accept the terms of service and submit. It will display 2 keys: the site key and the secret key.

Then, in Cyclos configuration form, under the CAPTCHA section, select reCAPTCHA v2 as provider and paste both keys to the corresponding fields.

IMPORTANT! To make the reCAPTCHA works for the mobile app, please ensure the web server doesn't send the following HTTP response headers for the path /mobile-recaptcha:

  • X-Frame-Options;

  • X-XSS-Protection;

  • Content-Security-Policy.

1.3.11. External content storage

Cyclos handles stored files, such as images, documents and binary custom field values. By default, they are stored in the database. However, for large systems, having files in the database will complicate backups, which will be very large. Also, having files in the database can be suboptimal in terms of performance. However, Cyclos also provides other storage types.

Storage types

Cyclos comes with four implementations out of the box:

  • Database: the content is stored in conjunction with all data in the database. This is the default implementation;

  • File system: the content is stored outside the database in specific paths;

  • Amazon S3: Amazon Simple Storage Service, the content is stored outside the database in specific buckets. As DigitalOcean’s storage is compatible with S3, it can also be used;

  • Google Cloud storage: the content is stored outside the database in a specific bucket.

Besides the built-in implementations, you can create your own custom implementation. To do that, you can create a Java class implementing org.cyclos.impl.storage.StoredFileContentManager

Some storage types support specifying custom directories. They can be set to provide different permissions for physical access of those files. All file storages, except for database, support storage directories.

All file storage settings are configured in cyclos.properties. First, set the cyclos.storedFileContentManager property to specify which storage should be used. Then set additional properties according to the type. Each storage type is described below.

File system storage

  • cyclos.storedFileContentManager: Set to file;

  • cyclos.storedFileContentManager.rootDir: The root directory where the files will be stored;

  • cyclos.storedFileContentManager.directories: comma separated list of folder (storage directory) names that will be created as children of the rootDir and where individual documents and image/file custom field values can be stored;

  • cyclos.storedFileContentManager.maxSubDirs: the maximum number of directories to be created below the root directory or a specific storage directory where the content will be stored.

Amazon S3 storage

This can also be used for DigitalOcean Spaces.

  • cyclos.storedFileContentManager: Set to s3;

  • cyclos.storedFileContentManager.bucketName: the name of the default bucket that will be created (if it doesn't exist) and where the content will be stored;

  • cyclos.storedFileContentManager.regionName: the name of the default region where the buckets will be created;

  • cyclos.storedFileContentManager.directories: comma separated list of bucket (storage directory) names where individual documents and image/file custom field values can be stored;

  • cyclos.storedFileContentManager.accessKeyId: the AWS access key;

  • cyclos.storedFileContentManager.secretAccessKey: the AWS secret access key;

  • cyclos.storedFileContentManager.serviceEndpoint: Not needed for Amazon, but used when using a S3-compatible system, such as DigitalOcean Spaces;

  • cyclos.storedFileContentManager.signinRegion: Not needed for Amazon, but used when using a S3-compatible system, such as DigitalOcean Spaces.

If you need to create a bucket in a different region than the default one, then you need to define a property of the form: cyclos.storedFileContentManager.regionName.<bucket_name>=specific_region_name

Google Cloud storage

  • cyclos.storedFileContentManager: Set to gcs;

  • cyclos.storedFileContentManager.bucketName: the name of the default bucket that will be created (if it doesn't exist) and where the content will be stored;

  • cyclos.storedFileContentManager.credentialsFile: path to JSON key file downloaded when creating the service account.

Storage migrator utility class

When you perceive the need to move the stored files out of the database to an external storage, you can use a utility, which is shipped together with Cyclos, to perform the migration.

First do a backup of your database, in case it needs to be restored.

The recommended strategy for running the migration tool in production is as follows:

  1. First test this in a test environment with a copy of the live database. See Setup a test environment for more details on this;

  2. Plan this at night or at a moment that the system is not heavily used;

  3. Run the migration while Cyclos is running. It will migrate all existing images in batches, using several parallel threads (one per available CPU core);

  4. When the migration finishes, stop Cyclos;

  5. Run the migration again to make sure any newly uploaded file is also migrated;

  6. Update your cyclos.properties, setting cyclos.storedFileContentManager file with the new value;

  7. Restart Cyclos.

Note: After step 3, errors will show up while accessing already migrated files, because those are no longer stored where Cyclos looks for them (i.e., the database). However, the system will be online for all other operations.

To run the utility, you first need to update the cyclos.properties file, leaving cyclos.storedFileContentManager with your current value (probably db), but already preparing all the other properties for the new storage. For example, when migrating to gcs, set both cyclos.storedFileContentManager.bucketName and cyclos.storedFileContentManager.credentialsFile. Or when migrating to s3, set cyclos.storedFileContentManager.bucketName, cyclos.storedFileContentManager.regionName, cyclos.storedFileContentManager.accessKeyId and cyclos.storedFileContentManager.secretAccessKey.

Then, in a console in the server running Cyclos, go to the Cyclos installation directory and run the following command:

java -cp "WEB-INF/classes:../../lib/*:WEB-INF/lib/*" \
    org.cyclos.impl.storage.utils.StoredFileContentMigrator

If Tomcat’s lib directory is located elsewhere, replace the :../../lib/*: part with :/path/to/tomcat/lib/*:. If that’s the case, you should see an error like java.lang.ClassNotFoundException: javax.servlet.ServletContext.

You will be presented with the usage help. Then, to finally migrate to the new storage, repeat the previous command, appending the value of the new storage (file, s3 or gcs) as an argument to the command.

1.3.12. Read data from a hot standby

When a PostgreSQL hot standby server is used, the master server can be offloaded by directing read-only queries to the hot standby server. This can be obtained by specifying additional datasource properties, with the cyclos.datasource.readOnly prefix. For example:

cyclos.datasource.provider = hikari
cyclos.datasource.dataSourceClassName = org.postgresql.ds.PGSimpleDataSource
cyclos.datasource.dataSource.portNumber = 5432
cyclos.datasource.dataSource.serverName = database-master
cyclos.datasource.readOnly.dataSource.serverName = database-replica # This is the overridden property
cyclos.datasource.dataSource.databaseName = cyclos4
cyclos.datasource.dataSource.user = cyclos
cyclos.datasource.dataSource.password = cyclos

All properties specified for the regular datasource can also be specified for the read-only datasource. The default value of all read-only datasource properties are those set for the regular datasource. You can then override only the ones that are different. So, in this example, the same Hikari dataSource.databaseName, dataSource.user, and so on, will be used for both data sources.

An important point for consistent queries between the master and the replica servers is the synchronous_commit setting in the master server. It should be set to either on (the default) or remote_apply. Take care that the setting is only used when synchronous_standby_names is also set. The remote_apply setting will wait until the data is visible for queries in the replica before the commit ends. This will introduce an additional delay in read-write transactions, but ensures data consistency. Without it, it might happen that after saving some data, the next request to read the same data will fail. This is a trade-off between consistency and performance. However, the odds for this are low, because there are the delays between the first request’s response being read by the client and the second request being sent and processed by the server. It is very like that such times are higher than the replica server time to apply the changes. This policy should be defined by the system administrator.

1.3.13. Using OpenSearch

For large systems, the time required for searching users, advertisements or transactions can be unacceptable when searching the database. Such queries can be complex, using full-text keywords or geo-distance filters.

For such systems, it is advised to use OpenSearch as search provider. For details on how to configure Cyclos with OpenSearch, refer to this section.

1.3.14. Logging

By default, Cyclos logs access to services, as well as background tasks, to files. But besides logging to files, it is also possible to log to an external PostgreSQL database. Logging to an external database has some benefits, especially being easier to find data when needed.

Logs are, by default, asynchronous, so the requests are not delayed until the log is written. This is controlled by the cyclos.log.threads property, which sets the number of threads used to concurrently write logs. However, if the server crashes, some log entries may be lost. If the number of threads is set to zero, logs will be synchronous, delaying each response. This guarantees that all logs are written before returning the response, but can have a high impact on the system throughput.

To choose the log provider, the cyclos.log setting should be changed in cyclos.properties. There are also additional settings for specific log providers:

Logging to files

  • cyclos.log: Set to file;

  • cyclos.log.dir: The directory where to write logs. Supports the following variables:

    • %t: The system temporary directory;

    • %w: The web application directory;

    • %n: The network internal name (or global).

  • cyclos.log.maxFiles: The log files are rotating. This setting indicates the maximum number of files per network.

Logging to an external database

It is important that the PostgreSQL database used for storing logs isn’t the same as the main Cyclos database. The database itself must be created before starting Cyclos.

  • cyclos.log: Set it to db;

  • cyclos.log.datasource.: All properties inside this prefix are used to create a datasource, and are the same options available to the regular cyclos.database. settings.

If using a database log, make sure to set the maximum number of connections equal to the number of threads set in the cyclos.log.threads setting. When using synchronous logging, the maximum connections to the log database will also limit the number of concurrent requests, so, extra care is needed.

A final note on the log database: there are currently 2 tables: service_logs and task_logs. The service_logs store a row each time a client calls any Cyclos service, while the task_logs stores a row for each background task execution. To make insertion of rows as fast as possible, the tables have no indexes, and will store parameters and results as the PostgreSQL’s JSON type, which has minimal impact on inserts (comparing to JSONB).

However, for searching data, the JSONB type (binary JSON) is more efficient, and supports indexing. When searching the table with too many logs, instead of searching directly on service_logs, it is recommended to create a new table, and query it instead. This new table should have the same columns as service_logs, but with JSONB columns instead. Also, add indexes according to your query.

A final note on log tables is that they tend to quickly grow in size, so you may need to periodically (according to the database data volume) move old data to another database in order to not impact the logging performance. Also, don’t forget to vacuum the table after deleting old records.

1.3.15. Hosting the frontend separately from Cyclos

For instructions on how to host the frontend separately from Cyclos please see the project instructions page at GitHub.

1.3.16. Data retention period / archiving

Very large systems may produce a huge amount of transactions per day. For such systems, having the full data over many years in the production database may be challenging, as it increases the database space, makes it harder to backup the database, increases the OpenSearch index size, etc.

Starting with Cyclos 4.16, such systems can define a data retention period - for example, 2 years - and configure Cyclos to only consider transfers, transactions and balances within this period. Then, an external application (provided by STRO upon request) connects to the Cyclos database, backups data in an external database, and then physically deletes such data in the Cyclos production database.

For understanding which data is archived up, it is important to understand the structure of transfers vs transactions in Cyclos. A quick review: a transfer is an actual balance transfer between 2 accounts, while a transaction is an intent of transferring balances between accounts. There are 2 kinds of transactions: payments, which generate one or more transfers once processed, and others which generate a payment (that will generate transfers). Payments are of 3 kinds: direct, which generate a single transfer once processed; scheduled, which splits the total amount in one or more installments (and each installment generates a transfer when processed); and recurring, which repeats the same amount on every processed occurrence (generating a transfer per occurrence). Other kinds of transactions are the ones which generate a payment when processed: payment requests, tickets and external payments.

These are the data effectively copied to the archive:

  • Transfers, together with information from the corresponding transaction, such as channel, description and custom values;

  • Account balances. Balances are actually calculated in the side of the archive application and stored for each account per month it had transfers.

Carefully consider that the following data will be permanently lost after archiving:

  • Transfer status (status flows). Those statuses typically represent a current follow-up of the transfer and don’t make sense from a historical point of view. They are not archived. The log of status changes is also not archived;

  • Information of payment requests, external payments and tickets;

  • Scheduled / recurring payment installment / occurrence information;

  • Amount reservations details (although those are not currently visible in Cyclos, only stored in the database);

  • Binary (file / image) transaction custom field values. When archiving, the corresponding files (for example, stored in Amazon S3 or Google Cloud Storage) are physically removed.

Note: When physically removing transfers after archiving, it is then possible to permanently remove transfer types or transaction custom fields which were only referenced from archived transfers. However, this is not recommended, because the same data will be used to show the details of the archived transfer in Cyclos.

In order to enable archiving, you have to set the following in cyclos.properties:

  • cyclos.archiving.months: The number of months to keep live transactions in Cyclos. Required for archiving;

  • cyclos.archiving.url: The URL used to contact the archiving application. Required for archiving;

  • cyclos.archiving.user: The HTTP username used on connections to the archiving application;

  • cyclos.archiving.password: The HTTP password used on connections to the archiving application;

  • cyclos.archiving.trustAllCerts: If the connection uses HTTP, this setting allows using self-signed certificates.

Also, Cyclos has a permission which can be enabled in administrator permissions / products (only visible when archiving is enabled in cyclos.properties) for administrators to query archived data. This functionality is available from the account history page itself, with a button to query for archived data, and fetches data from the external application via API.

This archiving functionality should not be confused with specific archiving settings in cyclos.properties, which are: cyclos.purgeMessagesOnTrash.days, cyclos.archiveImports.days, cyclos.archiveAccountFees.days and cyclos.archiveBulkActions.days. Those settings control the plain removal of data periodically from the Cyclos database.

Archiving is performed in a monthly basis. A recurring task will check daily whether the current month’s archiving was performed. If not (for example, the server was down on the first day), it will calculate the new archiving date and update the archived balances on all accounts. After all accounts archived balance are calculated, Cyclos will notify the archiving application to start the external archive.

When enabled, information about the current archiving status can be seen from the system monitor page.

Note: Database archiving is usually only needed for very large databases. Before implementing database archiving other measures can be taken. For example, storing images, files and documents outside the Cyclos database will considerably diminish the database size, and this is a common first step before implementing database archiving. See this section for more details.

1.4. Maintenance

1.4.1. Backup

All data in Cyclos is stored in the PostgreSQL database. Making a backup of the database can be done using the pg_dump command. In terms of files, you only need to backup the cyclos.properties and, if customized, the hazelcast.xml configuration files.

The database can be backed up manually as follows:

pg_dump --username=cyclos --password -hlocalhost cyclos4 > cyclos4.sql

Note: in this example the name of the database is cyclos4, the username cyclos and the command will prompt for the password of the cyclos user.

1.4.2. Restore

To restore a database dump to another database, first create the new database (in this example, cyclos4) and grant permission to the user to manage it (in this example, the user is cyclos). Supposing the file containing the dump is cyclos4.sql in the current directory, the following command will import the data:

psql --username=cyclos --password -hlocalhost cyclos4 < cyclos4.sql

1.4.3. Backup / restore large databases

When the database is very large (especially if it has a lot of images) it is possible to use a custom format for the dump file, which makes the dump file smaller. To use it, backup with the following command:

pg_dump --username=cyclos --password -Fc -hlocalhost cyclos4 > cyclos4.sql

To restore the dump, another command needs to be used as well:

pg_restore --username=cyclos --password -Fc -hlocalhost -d cyclos4 cyclos4.sql

Note: for larger databases, it is highly advised to store binary files outside the database, specially in a managed storage service such as Amazon S3 or Google Cloud Storage. For more details, see External content storage.

1.4.4. Reset admin password

If you lost the password of your global administrator, it is still possible to update the value on the database directly. To reset the password to 1234, run the following sql in the PostgreSQL query tool (psql). Replace the 'admin' username by the username of your global administrator.

update passwords
  set value='$2a$10$yM.uw9jC7C1DrRGUhqUc3eSR6FCJH0.HdDt3CJs8YL56iATHcXH7.'
where user_id = (select id from users where username='admin')
  and status = 'ACTIVE'
  and password_type_id in (select id from password_types where input_method = 'TEXT_BOX' and password_mode = 'MANUAL');

1.4.5. Restore applied themes

If you are locked out of the system because an applied theme is failing compilation because of some error, you can run the following SQL to apply a built-in theme to both guests and logged users. After running it, only the global default URL will have an applied theme - all others will just inherit them.

with default_theme as (
select min(id) as id
    from themes
    where type = 'MAIN_WEB'
    and file_name is not null)
update configurations set
guests_theme_id = case when parent_id is null then default_theme.id else null end,
users_theme_id = case when parent_id is null then default_theme.id else null end
from default_theme;

1.4.6. Share database dumps

If Cyclos development team or a third party asks you to share the database with them, specially for bug isolation and support, it is vital for security that sensitive configurations, passwords and personal data are removed from the database.

The passwords in Cyclos are hashed with one of the strongest algorithms available (Bcrypt), but still passwords can be theoretically recovered using brute force (although very unlikely). If the database falls into the wrong hands, some users might get compromised. Therefore, it is always recommended to follow this procedure before sharing the database with other parties:

  • Make a dump of the database (see Backup);

  • Restore the database in another (temporary) database, so the data can be changed without risking changing live data (see Restore);

  • Run the following series of SQL commands, preferably storing all these in a script file, so it can be replayed whenever needed:

Reset all passwords to '1234':

update passwords
set value = '$2a$04$rDPKseEiJhYdjx9RogW2tuzNX4TKG1wcE79ooEXiA5.mJF.ooZY/2'
where status <> 'OLD'
and password_type_id in (
    select id
    from password_types
    where input_method = 'TEXT_BOX'
    and password_mode = 'MANUAL'
);

Remove sensitive user information:

delete from sessions;
delete from phones;
delete from addresses;
delete from user_fcm_tokens;
update users
  set email = concat(username, '@localdomain')
  where email is not null;

If you have a custom field with private information, such as ID card, you can also remove the values with (adjust the <internal_name> value):

delete from user_custom_field_values
  where field_id = (
    select id
    from user_custom_fields
    where internal_name = '<internal_name>');

Remove sensitive configurations:

update configurations set
    api_url = null,
    smtp_from_address = 'noreply@localhomain',
    smtp_host = 'localhost',
    smtp_port = '25',
    smtp_security = 'NONE',
    smtp_user = null,
    smtp_password = null,
    sms_enabled = false,
    sms_gateway_url = null,
    sms_username = null,
    sms_password = null,
    sms_headers = null,
    firebase_private_key = null,
    map_server_api_key = null,
    map_browser_api_key = null,
    captcha_recaptcha_key = null;

You may also have API keys, passwords, etc. stored in custom script parameters. If this is the case, make sure to also change that.

Finally, create a dump of the temporary database. This dump and then be sent to the third party.

1.4.7. Setup a test environment

For running a production system, it’s of vital importance to run not only a live instance but also, a test instance. Besides running a test instance, many systems will benefit from also having a development instance. When updating Cyclos, creating new scripts, or doing any change with impact to the system, we recommend to always try it out on the test instance first. If all goes well on the test system, the change can be made on the live system too.

We recommend creating a script that copies the database from a live instance backup and, before the test instance is started, imports that dump and removes all vital information from the database. It’s very important that customers won’t receive emails, SMS messages or mobile app push notifications from a test instance.

We recommend first removing all user-related private information and production-specific API keys. For this, please refer to the Share database dumps section.

Also, the following database commands are recommended, and can be adjusted to better fit your needs. Adjust the variables presented between < and >.

update configurations set
    root_url = '<https://test.instance.url>',
    email_unique = false,
    application_name = case
        when application_name is null then null
        else 'Test - ' || application_name end;
delete from admin_notif_settings_authorizable_payments;
delete from admin_notif_settings_payments;
delete from admin_notif_settings_fwd_message_categories;
delete from admin_notif_settings_user_alerts;
delete from admin_notif_settings_system_alerts;
delete from admin_notif_settings_user_groups;
delete from admin_notif_settings_fwd_message_categories;
delete from notification_type_settings;
delete from admin_notif_settings_external_payments_expired;
delete from admin_notif_settings_external_payments_failed;
delete from admin_notif_settings_voucher_configurations_buying;
delete from admin_notif_settings_voucher_configurations;
delete from user_account_notification_settings;
delete from notification_settings;
update static_contents set content='<div style="color: red; font-size: 28px; margin-bottom: 20px; font-weight: bold;">Test instance</div>' || content where subclass='HEADER';

Finally, we recommend using a test SMTP server, in this way a client could never accidentally get an email message. An easy and free solution for this is Mailtrap, but there are also self-hosted solutions available, such as Mailhog. Here’s an example for Mailtrap:

update configurations set
    smtp_from_address = 'noreply@localhomain',
    smtp_host = 'smtp.mailtrap.io',
    smtp_port = '465',
    smtp_security = 'NONE',
    smtp_user = '<your-mailtrap-user>',
    smtp_password = '<your-mailtrap-password>';

After all these adjustments, you can start the test instance.

1.4.8. Remove network data

A common practice for a first-time configuration of Cyclos, specially with a complex structure for accounts, configurations, products and groups, is to configure all the system, and create some test users and payments. However, after finishing configuration, it might be desirable to remove all users and transfers (payments) from that network, leaving only administrators and configurations. Alternatively, it might be desirable to completely delete an entire network.

An interactive utility is included in Cyclos, which can be used for both cases. Please, be advised to perform a full database dump before running the utility, and have Cyclos stopped before running it.

To run the utility, go to <tomcat_home>/webapps/<instance_name> directory and execute the following:

java -cp "WEB-INF/classes:../../lib/*:WEB-INF/lib/*" \
    org.cyclos.db.DeleteNetworkData

If Tomcat’s lib directory is located elsewhere, replace the :../../lib/*: part with :/path/to/tomcat/lib/*:. If that’s the case, you should see an error like java.lang.ClassNotFoundException: javax.servlet.ServletContext.

Then follow the instructions presented on the console. When a lot of data is removed, it is recommended to run a full vacuum in the database. This operation might take a while. To run it, execute the following command:

$ vacuumdb --full $DATABASE_NAME

2. Full-text searches

This chapter covers how full-text searches work in Cyclos, and how to fine-tune them.

Full-text searches allow retrieving documents using its words, returning documents that match a given textual query (often related as keywords in Cyclos). The full-text engine processes words both when indexing (calculating the words on documents) and querying (transforming an input text in a way it matches indexed documents). Some examples of such processing include:

  • Removing stop words - words which are too common in a given language, and likely be contained in multiple documents. In English, 'a', 'the' and 'is' could be example of stop words;

  • Changing words to a common form, or stemming. For example, in English, 'sailing', 'sailed', 'sailor' could all be stored as 'sail'.

Currently, the following entities are searched with full-text queries when using keywords:

  • Users: The profile fields which are set in the user products (or group’s permissions in case of administrators) marked to include in user keywords will be searched. Also supports geo-distance searches;

  • Advertisements: The advertisement title, description and custom fields, plus the user (owner) profile fields which are set in the user products marked to include in advertisements keywords will be searched. Also supports geo-distance searches;

  • Records: The record custom fields, plus the user profile fields which are set in the user products (or group’s permissions in case of administrators) marked to include in record keywords will be searched;

By default, Cyclos uses the native PostgreSQL’s full-text indexing capabilities. Also, for geospatial distance filters, Cyclos uses PostGIS. However, for large systems, such queries can present an unacceptable slow performance. In such cases, the system should use an external OpenSearch server.

As the PostgreSQL native query syntax is too much formal for end users, a query preprocessor is included in Cyclos, such that the following variants are supported:

  • a b: The value must have words that either start with a or b;

  • a +b: The value must have words that start with both a and b;

  • a -b: The value must have words that start with a and no words that start with b;

  • "a b": The value must have exactly a followed by b, in this exact order;

  • Also, parenthesis can be used to group expressions, such as ( (a b) +(c -d) ).

The fulltext dictionary used by Cyclos in PostgreSQL is a simple one, only performing the unaccent and ignoring case operations (so, for example, acao matches Ação). For more advanced operations, such as analyzing the text according to the configured language, OpenSearch should be used.

2.1. Using OpenSearch

For large systems, the time required for searching users, advertisements or transactions can be unacceptable when searching the database. Such queries can be complex, using full-text keywords or geo-distance filters.

For such systems, it is advised to use OpenSearch as search provider. OpenSearch is a fork of the well-known Elasticsearch system. OpenSearch was created because Elasticsearch 7.11 onwards is no longer open source software (OSS), and cannot be used by cloud providers, such as Amazon. Because many Cyclos systems are hosted on Amazon and use it’s managed Amazon OpenSearch Service, it makes more sense for Cyclos to support OpenSearch than Elasticsearch. OpenSearch 1.1 and Elasticsearch 7.10 are virtually the same, but it is expected that both systems divert more and more with newer releases.

The OpenSearch server / cluster needs to be deployed in a server accessible to Cyclos via its REST API. Then, in cyclos.properties, add the following settings:

  • cyclos.searchHandler = opensearch: This enables the OpenSearch integration. Make sure to comment out or remove the line cyclos.searchHandler = db;.

  • cyclos.searchHandler.host: One or more (comma-separated) hosts (protocol://hostname:port) of the OpenSearch server;

  • cyclos.searchHandler.pathPrefix: Path within the host on which the OpenSearch server responds to;

  • cyclos.searchHandler.user: Optional user for HTTP basic auth;

  • cyclos.searchHandler.password: Optional password for HTTP basic auth;

  • cyclos.searchHandler.shards: How many shards the indexes should be split in;

  • cyclos.searchHandler.replicas: How many replicas per shard;

  • cyclos.searchHandler.trustAllCertificates: By default, OpenSearch is configured with a self-signed certificate to handle HTTPS connection. If this setting isn’t set to true, Java will reject such a connection, because the certificate issuer cannot be validated.

Once set-up, when restarted, Cyclos will attempt to find the following OpenSearch indexes, and, if not found, will create them and index all entities in the database: users, ads, records, transactions, transfers and installments. Indexing will be executed on the background, and it can take several minutes to index all data, depending on the database size.

Textual searches (referred to as keywords in Cyclos) are passed to OpenSearch using Simple Query String syntax, which is like the one employed by Cyclos when searching on the database, but more powerful.

2.2. Language processing

The actual language processing (such as removing stop words and stemming text) is performed only on the following fields: advertisement title and description, and custom fields whose setting "Value match" is set to "Language". Note that the "Value match" field will only show up in custom fields when OpenSearch is used. When handling searches in the database, only the checkbox "Use exact matching on search filters" will show up.

Language-analyzed fields are stored multiple times in the index: one using OpenSearch Standard Analyzer (to prevent mismatches when searches are performed by users which use other languages), and another one for each language the owner of the data can have. So, for example, an advertisement owned by a user whose configuration allows English, Portuguese and French will have the field stored in all such analyzers, plus the standard.

When searching for the data, Cyclos uses all languages the logged user has, plus the standard analyzer. So, following the same previous example, if a logged user had only the French language, it would search in both the standard analyzer field, plus the French one. And if another user could see that advertisement, but have only the Spanish language, they would search in both Spanish (in which the no data exist) and in standard, and would ultimately find the advertisement using the standard analyzer.

2.3. Reindexing

Being a separated data store, the data on PostgreSQL database and on OpenSearch might become de-synchronized. The database is always considered correct, and is the trusted store. The data on OpenSearch is updated automatically, as soon as the corresponding entity is modified on the database. But it might happen that an update request fails, or that the OpenSearch server is offline for some time.

Important: If you have modified a profile field (its value or the internal name) then you must reindex users, advertisements and records.

To handle these cases, Cyclos offers methods that can be executed by scripts, directly through the menu System > Tools > Run script. Here are some examples:

Reindex ALL data on ALL indexes:

searchHandler.reindex()

Reindex ALL data on a specific index:

userSearchHandler.reindex()
adSearchHandler.reindex()
recordSearchHandler.reindex()
transactionSearchHandler.reindex()
transferSearchHandler.reindex()
installmentSearchHandler.reindex()

Other examples:

// Reindex a single user
import org.cyclos.entities.users.User
def user = conversionHandler.convert(User, 'loginName')
userSearchHandler.index(user)
// Reindex a single advertisement by external (masked) id (would be similar to records)
import org.cyclos.entities.marketplace.BasicAd
def ad = entityManagerHandler.find(BasicAd, unmaskId(123456789L))
adSearchHandler.index(ad)

3. Web services

Here you will find information on how to call Cyclos web services from 3rd party applications. Web services must not be invoked by Cyclos scripts, refer to this section for more information.

Cyclos 4 provides distinct web service interfaces: the REST API, custom web services and WEB-RPC. In all cases the security layer is exactly the same (hence, both grant exactly the same permissions), and users are authenticated in the same way, as described below.

3.1. REST API

This API is implemented with REST concepts in mind, such as using proper HTTP verbs (GET, PUT, POST, DELETE), using JSON data for input and output. It should be relatively easy for developers to leverage existing knowledge when using it. It is documented using OpenAPI, which enjoys rich documentation and tooling support. For example, there are OpenAPI generators that can generate clients for distinct languages / frameworks.

A detailed reference documentation is available online on each Cyclos installation, at <cyclos-root-url>[/network]/api. You can also refer to the Cyclos Demo API. It is possible to disable the API reference documentation page by setting cyclos.rest.reference = false in cyclos.properties.

The REST API contains a subset of the Cyclos functionality. Most notably, system management and content management are not part of the API. Most user-facing operations, however, are available via the REST interface. New functionality will be added on demand, in a cautious manner, as each path, parameter and data model needs to be planned to fit the target architecture.

The REST interface is the preferred API for 3rd party clients to connect to Cyclos, as it should be relatively stable between Cyclos releases. Cyclos follows the following policy: any breaking changes / removals in the API will be kept as deprecated for 2 versions, then removed in the next one. For example, an operation can be deprecated in Cyclos 4.16 and 4.17, to be finally removed in 4.18.

3.2. Custom web services

It is also possible to create custom web services, which run a script on every invocation. Custom web services can be configured to be executed as a guest, with a fixed HTTP user / password or as a Cyclos user (using the same authentication as other web services).

After creating a custom web service script, you have to create a custom web service entity in 'System > Tools > Custom web services'. There you will find the 'Url mappings' field, which contains the path which the script should be available. The endpoint for the custom web service will be <cyclos-root>/run/<mapping>.

Also, take into account that custom web services executed as a logged user must also be granted to users through a product in 'System > User configuration > Products (permissions)'.

3.3. WEB-RPC

WEB-RPC provides direct access to the internal service interfaces in Cyclos, via web services. It is not recommended for 3rd party applications to use WEB-RPC directly, but the REST API instead.

Starting with Cyclos 4.16, we will no longer publish a PHP library for services, nor detailed information on WEB-RPC in general. For historic purposes, the documentation is kept at https://cyclos.org/documents/legacy-web-rpc.html.

3.4. Authentication in web services

Regardless of the web service interface (REST, WEB-RPC or custom), users are authenticated either as user / password (stateless), logging-in with a session (stateful) or using access clients (stateless). The way authentication data is passed from client to server depends on whether the clients are using a web service or a client API (Java or PHP).

3.4.1. User and password

In this mode, a principal (user identification method), which can be the login name, e-mail, mobile phone, custom field, account number or token value (card number), depending on the channel configuration, is sent on each request together with the password (live systems must always be over HTTPS, so should be secure). The drawbacks are that the username and password need to be stored in the client application, and changing the password on the web (if the same password type is used) will make the application stop working.

3.4.2. Login with a session

In this mode, first a request authenticated with user and password is made with POST /api/auth/session. It returns a session token, as well as other information of the logged user. Subsequent requests should pass this session token instead in the subsequent requests, via the Session-Token HTTP header. To end a session (logout), a request authenticated with the session token must be performed with DELETE /api/auth/session.

3.4.3. Access clients

Access clients can be configured to prevent the login name and password to be passed on every request by clients, decoupling them from the actual password, which can then be changed without affecting the client access. It also improves security, as each client application has its own authorization token, which can be individually blocked or revoked.

To configure access clients, first a new identification method of this type must be created by administrators in 'System > System configuration > User identification methods'. Then, in a member product of users which can use this kind of access, permissions over that type should be granted. Finally, the user (or an admin) should create a new access client in Cyclos main access, and get the activation code for it.

The activation code is a short (4 digits) code which uniquely identifies an access client pending activation for a given user. To use the access client, on the client application side (probably a server-side application or an interactive application), a request must be performed: POST /api/clients/activate, passing the username / password in a BASIC authentication and sending the activation code as the code query parameter.

The result will be a token which should be passed in subsequent requests using the HTTP header Access-Client-Token. The activation process should be done only once, and the token will be valid until the access client in Cyclos is blocked or disabled.

Here is an example which can be called by the command-line program curl:

curl https://<cyclos-root>[/network]/api/clients/activate?code=<4-digit code> \
    -u "<username>:<password>"

The generated token will be printed on the console, and should be stored on the client application to be used on requests.

Additionally, clients can improve security if they can have some unique identifier which can be relied on, and don’t need to be stored. For example, Android devices always provide a unique device identifier. In that case, this identification string can be passed at the moment of activation, and will be stored on the server as a prefix to the generated token. The server will return only the generated token part, and this prefix should be passed on requests together with the generated token. The prefix is passed in the activation request as a query parameter prefix. So, for example:

curl https://<cyclos-root>[/network]/api/clients/activate?code=<4-digit code>&prefix=XYZW \
    -u "<username>:<password>"

Imagining the server returns the fictional token ABCDEFG (the actual token is 64 characters long), the actual token that then should be passed to requests is XYZWABCDFG.

3.5. Channels

Channels can be seen as a set of configurations for access in Cyclos. There are some built-in channels, and additional ones can be created. The built-in channels that can use used on web services are:

  • Main web: The main web application. The internal name is main;

  • Mobile: The Cyclos (or another 3rd party) mobile application. The internal name is mobile;

  • Web services: Is the default channel for clients using any web service client. The internal name is webServices.

By default, the channel used on any web service (regardless of the interface or user authentication mode) is Web services. It is possible to specify another channel, for example, with third party web applications (handled as Main web) or third party mobile applications.

In such cases, the channel internal name must be passed on each request using the HTTP header Channel.

3.6. Configuring web services

For clients to invoke web services in Cyclos, the following configuration needs to be done on the server (as global or network administrator):

  • On 'System > System configuration > Configurations' menu, select the configuration used by users to go to the configuration details page;

  • On the 'Channels' tab, click on the 'Web services' channel row, to go to the channel configuration details page;

  • Make sure the channel is enabled. It can be allowed or disallowed by default. Click the edit icon on the right if the channel is not defined for this configuration. Then mark the channel as enabled, choose the way users will be able to access this channel (by default or manually) and the password type used to access the web services channel. You can also set a confirmation password, so sensitive operations, like performing a payment, will require that additional password;

  • For specific users, in the user profile page (as administrator), under the 'User management' box, click the 'Channels access' link;

  • On that page, make sure the 'Web services' channel is enabled for that user. Also, only active users may access any channel - on the profile page, on the same 'User management' box, there should be a link with actions like 'Enable / Block / Disable / Remove'. On that page, make sure the user status is 'Active';

  • A side note: If performing payments via Web services, make sure the desired 'Transfer type' is enabled for the 'Web services' channel. To check that, go to 'System > Account configuration > Account types' menu item. Then click the row of the desired account type, select the 'Transfer types' tab and click on the desired payment type (generated types cannot be used for direct payment). There, make sure the 'Channels' field has the 'Web services' channel.

3.7. External login

With the right configuration, it is possible to add a Cyclos login form to an external website, such as the company main website.

The user types in their Cyclos username and password in that form and, after a successful login, is redirected to Cyclos, where the session will be already valid, and the user can perform the operations as usual.

After the user clicks logout, or the session expires, the user is redirected back to the external website.

The following aspects should be considered:

  • It is needed to have an administrator whose group is granted the permission 'Login users via web services'. This is needed because the website will relay user logins to Cyclos, authenticated as that administrator;

  • The website needs to have that administrator’s username and password configured in order to make the web services call. Even better, you can configure an access client which will allow using a separated key instead of the username / password;

  • It is a good practice to create a separate configuration for that administrator. That configuration should have an IP address whitelist for the 'Web services' channel. Doing that, no other server, even if the administrator username / password is known by someone else, will be able to perform such operations;

  • The Cyclos configuration for users needs the following settings:

    • 'Redirect login to URL': This is the URL of the external website which contains the login form. This is used to redirect the user when his session expires and a new login is needed, or when the user navigates directly to some URL in Cyclos (as guest). In that case, the external website receives a parameter named returnTo that must be sent back to Cyclos without any modification after a successful login;

    • 'URL to redirect after logout': This is the URL where the user will be redirected after clicking Logout in Cyclos. It might be the same URL as the one for redirect login, but not necessarily.

  • Finally, the web service code needs to be created, and deployed to the website. Here is an example, which receives the username and password parameters, calls the web service to create a session for the user (passing his remote address), redirecting the user to Cyclos.

Cyclos plugin for WordPress

Cyclos provides a plugin for the well-known WordPress CMS system. It also serves as an example of the external login. For details, refer to https://wordpress.org/plugins/cyclos/.

Important notes

  • In case there is a wrong configuration for the Redirect login to URL setting, it won’t be possible anymore to login to Cyclos. In that case, if the configuration problem is within a network, it is possible to use a global administrator to login in global mode (using the <cyclos-root>/global URL), then switch to the network and fix the configuration. If the configuration error is in global mode, you can use a special URL to prevent redirect: <server-root>/global/login!noRedirect=true. However, this flag only works in global mode, to prevent end-users from using it to bypass the redirect;

  • Users should never have username / password requested in a plain HTTP connection. Always use a secure (HTTPS) connection. Also, just having an iframe with the form on a secure page, where the iframe itself is displayed in a non-encrypted page would still encrypt the traffic in the iframe, but browsers won’t show the page as secure (padlock icon). Users won’t see that page as secure, and could refuse to provide credentials in such a situation.

Creating an alternate frontend to Cyclos

It is possible to not only place a login form in an external website, but to create an entire frontend for users to interact with Cyclos. At first glimpse, this can be great, but consider the following:

  • It is a very big effort to create a frontend, as there are several Cyclos services involved, and it might not be clear without a deep analysis on the API which service / operation / parameters should be used on each case;

  • You will always have a limited subset of the functionality Cyclos offers. While you may initially think that only the very basic features are needed, there will inevitably be the need for more features, and the custom frontend will need to grow. By using Cyclos standard web interface, all this comes automatically.

Instead, a better approach could be to extend the new frontend (cyclos4-ui), which provides a modern interface and can be easily customized.

Nevertheless, some (large) organizations might find it better to provide their users a single, integrated interface. Also, some organizations develop all their services with mobile applications only.

4. Scripting

Cyclos scripting module provides an integration layer that allows connecting Cyclos to third party software, as well executing custom operations and scheduled tasks within Cyclos itself. The scripting module offers an easy way to customize and extend Cyclos. Scripts can access the full Cyclos services layer, which makes it a powerful feature. For security reasons only global administrators can manage scripts, as scripts can access any service, the database or even the physical server where Cyclos is running. Network administrators can then assign scripts to elements such as extension points (e.g. payment, user profile, advertisement), custom validations (for input fields), custom calculations (account fees, transaction fees), custom operations, scheduled tasks and many others.

Global admins can write scripts directly within Cyclos. Each script type has its own functions which have to be implemented. A network admin can choose from the available scripts and bind them to the actual entity (such as custom operations, extension points, etc.). The script can also use parameters, which are configured outside the script code. This avoids the need for modifying a script every time a new or different parameter is required.

The scripting language currently supported is Groovy. It is a powerful language that is very similar to Java, with a close to zero learning curve for Java developers. It is possible to write scripts that will be available in a shared script library, so that other scripts within the same context can make use of it. All scripts are compiled to Java byte-code, which minimizes the runtime performance impact.

Debugging scripts can sometimes be tricky, because the exact context is only available at runtime, and errors can be hidden. A good approach is to set cyclos.dumpAllErrors = true in cyclos.properties. This way, whenever an error is triggered, it is dumped to the application server (i.e., Tomcat) console. Also, see the section Debugging scripts for how to setup an IDE to develop scripts more comfortably.

4.1. Crucial considerations

Before writing any script code, please carefully consider the following points.

4.1.1. Database transactions

Scripts are executed in a database transaction. If there are any exceptions, the transaction is rolled-back. Otherwise, if all operations succeed, the transaction is committed. Scripts must never modify the current transaction to avoid database inconsistencies. This is a very hard situation to fix, requiring manual intervention in the database, and the Cyclos team cannot be held responsible in such cases.

Below are situations which can cause database inconsistencies.

Exception handling

If the script catches an exception without throwing another one, the current transaction will not be rolled-back, but committed instead. If there were other previous database operations which partially modified data, the final state of the database will be inconsistent. Here are some examples for this anti-pattern:

  • When performing a payment, a Payment entity is created. Then, if the payment doesn’t need authorization, also a Transfer entity is created, which effectively moves funds between the accounts. However, before creating the transfer, the available balance is checked. If the account doesn’t have enough balance, an exception is thrown. If silencing that exception, the database will have the Payment entity without the corresponding Transfer, which is an inconsistent state for Cyclos. This is a very hard situation to fix, requiring manual intervention in the database, and the Cyclos team cannot be held responsible in such cases.

  • PostgreSQL marks the current transaction to be rolled-back when there are any errors in database queries (for example, an integrity constraint validation). If that error is silenced, Cyclos may assume it is all OK, and even a success response can be returned to the client (for example, a transaction with a transaction number). However, that transaction was never persisted, because of the rollback. This situation can damage the system by, for example, notifying merchants that a payment is done, the physical product is sent, but the merchant never gets the payment.

Always catch the most specific exception type you need. And always, either rethrow another specific exception or, if you really need to silence an exception, you must mark the current transaction as rollback-only by calling transactionStatus.setRollbackOnly().

Manual transaction commit

Never manually commit the current transaction. For example, after committing the transaction, all locks are released, causing concurrency issues. Also, if the transaction fails, all previously commited data will be persisted but any database changes after that point will be rolled-back.

4.1.2. Inbound requests

Do not make any HTTP requests from scripts to Cyclos' own REST operations or custom web services. Though the REST services are well documented and easy to use, doing so will require opening a separated and parallel database transaction to handle the request, while leaving open the current transaction which is running the script. This can easily exhaust the database connection pool and even lead to situations where no more requests can be handled. Instead, scripts should only use the internal services and handlers.

4.1.3. Executor service

Do not create your own ExecutorService or ScheduledExecutorService unless you have a very good reason for it. Instead, used the shared scheduled executor which is available through the invokerHandler bounded variable, using its executorService property. It scales and shuts down threads as needed. Here is an example:

invokerHandler.executorService.submit {
    // This runs in another thread...
    println 'Task finished!'
}

Keep in mind that Groovy will resolve closures as Runnable by default, not as Callable. This means that the result of the Future.get() call will always be null, as the submit(Runnable) version is called in runtime, which always returns null. To actually use the result, you need to explicitly cast your closure to Callable, like this:

import java.util.concurrent.Callable

def future = invokerHandler.executorService.submit({
    // This runs in another thread...
    return 'My result'
} as Callable) // Note the parenthesis, as we're casting the argument, not the result

// This will block until the other thread finishes and print 'My result'
println future.get()

However, if you really need to create an ExecutorService, make sure to shut it down before the script ends. Failing to do so will cause the threads in the pool to be left open (depending on the executor type), which increases the number of system threads in each script execution. This will eventually crash the JVM due to an excessively large number of threads. So, enclose your code with a try / finally block and call ExecutorService.shutdown().

4.1.4. Long-running transactions

It is not a good idea to have a large job running in the same transaction. This is particularly dangerous for payments because each payment requires locking accounts*, and locks are only released on commit or rollback, causing the locks to be held for much time.

Instead, it is much better to break the job into smaller pieces. For example, you could move payments out of the main transaction, by using :

import org.cyclos.model.utils.TransactionLevel

invokerHandler.submitAsInParallelTransaction(sessionData, TransactionLevel.READ_WRITE) {
    // Code to make payment...
}

With this approach, locks will be released early, as the only thing that the separated transaction does is the payment, which would release the lock early.

Also, if you need to do some processing iterating a large amount of data (e.g. users, records, etc.) take a look at the custom background tasks, it is a much better option than manually processing each entity in the same transaction.

* How the account locks are performed can be changed through the payment type being used, please check the information for the "Lock origin account" and "Lock destination account" properties on the transfer type details page.

4.2. Variables bound to all scripts

When running, scripts have a set of bindings, that is, available top-level variables. At runtime, the bindings will vary according to the script type and context. On all cases, however, the following variables are bound:

  • scriptParameters: In the script details page, or in every page where a script is chosen to be used (for example, in the extension point or custom operation details page) there will be a textarea where parameters may be added to the script. They allow scripts to be reused in different contexts, just with different parameters. The text is parsed as Java Properties, and the format is described here. The library parameters are included first (if any), then the own script parameters (if any), then the specific page parameters. This allows overriding parameters at more specific levels;

  • globals: A shared ConcurrentMap (thread-safe) that can be used to store shared objects available to all scripts. When running in a cluster, each node will have its globals map instance. See this section for more details;

  • scriptHelper: An instance of org.cyclos.impl.system.ScriptHelper. Besides having the instance available, all its methods are automatically exported as closures on the default binding, making it possible to call its methods without using the scriptHelper. prefix. The ScriptHelper class contains some useful methods, such as:

    • wrap(object[, customFields]): wraps the given object in a Map<String,Object>, allowing reading / writing regular properties (getters / setters) and custom field values alike (via field internal names). Also, when setting values, they are automatically converted to the expected type when possible. See this section for more details.

    • bean(class): returns a Spring bean bean by type. The class reference needs to be passed. Available beans are all services, handlers and so on;

    • addOnCommit(callback) and addOnRollback(callback): Add callbacks to be executed after the main database transaction ends, either successfully or with failure. Can only be used in read-write transactions (transactionLevel.readOnly is false). Be aware that those callbacks will be invoked outside any transaction scope within Cyclos, so things like 'sessionData.loggedUser' won’t work (because it requires retrieving the User object from the database). However, it is more efficient, as no new database access needs to be done. This is mostly useful to notify an external application that some data has been persisted in Cyclos (after we’re sure that the data is persistent). Keep in mind that there is a (very) small chance that the main transaction is committed / rolled back but then the server crashes, and the callback wasn’t yet called. So, when synchronizing with external systems, it is always wise to do some form of timeout / recovery mechanism;

    • addOnCommitTransactional(callback) and addOnRollbackTransactional(callback): Same as the non-transactional counterparts, but the callback is executed in a new read-write transaction;

    • addOnAfterEnd(callback): Adds a callback to be executed after the current transaction ends. The callback receives a boolean flag. When in a read-only transaction, the flag will be null. When in a read-write, indicates whether the original transaction was committed (true) or rolled-back (false). The callback itself is executed outside a transaction;

    • addOnAfterEndTransactional(callback): Same as the non-transaction counterpart, but the callback is executed in a new read-write transaction;

    • addOnBeforeEnd(callback): Adds a callback to be executed right before the current transaction ends. The callback is executed in the same current transaction;

    • maskId(id) and unmaskId(id): In Cyclos the internal database ids are not visible to the clients, because of security reasons. The ids used in the web application or used in our web services are therefore always masked / obfuscated. These methods apply or remove the mask to the id.

  • sessionData: Contains information about the currently authenticated user (if any), as a org.cyclos.impl.access.ScriptSessionData;

  • entityManager: The JPA entity manager bound to the current transaction;

  • transactionLevel: The current org.cyclos.model.utils.TransactionLevel. When transactionLevel.readOnly is true, scripts should not modify the database in any way, or the current transaction will fail;

  • transactionStatus: The current transaction status, as org.springframework.transaction.TransactionStatus. If ever doing a script that silences an exception, the setRollbackOnly() method must be called to rollback the current transaction, avoiding an inconsistent database state. See this warning for more information;

  • formatter: A org.cyclos.impl.utils.formatting.FormatterImpl instance configured with the current user settings;

  • objectMapper: A com.fasterxml.jackson.databind.ObjectMapper which can be used to encode / decode objects to JSON strings;

  • jdbc or jdbcTemplate: org.springframework.jdbc.core.JdbcTemplate which can be used to perform native SQL queries using positional parameters;

  • namedJdbc or namedParameterJdbcTemplate: org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate which can be used to perform native SQL queries using named parameters;

  • rest or restTemplate: org.springframework.web.client.RestTemplate which can be used to perform REST HTTP requests;

  • Service implementations: All *ServiceLocal instances are bound via simple names, starting with lowercase characters, without the 'Local' suffix. For example, org.cyclos.impl.users.UserServiceLocal is bound as userService;

  • Security layer: All *ServiceSecurity instances are bound via simple names, starting with lowercase characters. Specially useful for custom web services, which have no security layer. For example, org.cyclos.security.users.UserServiceSecurity is bound as userServiceSecurity;

  • Internal handlers: All *Handler instances are bound via simple names, starting with lowercase characters. For example, org.cyclos.impl.access.ConfigurationHandler is bound as configurationHandler.

4.3. General tips for scripts

4.3.1. Using custom external dependencies using Groovy Grape

Groovy provides the Grape functionality to fetch Java dependencies in a Maven repository. All you need is to find the dependency in https://mvnrepository.com/, click on the version and copy the content of the Grape tab.

Using Grape to connect to a SFTP server

Here is an example of a library script which uses the a fork of the JSch library to download a file from a remote SFTP server to the local filesystem. The original JSch project was discontinued, and no longer works with modern encryption algorithms, hence, the fork.

@Grab(group='com.github.mwiede', module='jsch', version='0.2.8')

import com.jcraft.jsch.ChannelSftp
import com.jcraft.jsch.JSch
import com.jcraft.jsch.SftpException

class SftpHelper {
    static final CODE_TO_MSG = [
        (ChannelSftp.SSH_FX_OK): 'OK',
        (ChannelSftp.SSH_FX_EOF): 'EOF',
        (ChannelSftp.SSH_FX_NO_SUCH_FILE): 'NO_SUCH_FILE',
        (ChannelSftp.SSH_FX_PERMISSION_DENIED): 'PERMISSION_DENIED',
        (ChannelSftp.SSH_FX_FAILURE): 'FAILURE',
        (ChannelSftp.SSH_FX_BAD_MESSAGE): 'BAD_MESSAGE',
        (ChannelSftp.SSH_FX_NO_CONNECTION): 'NO_CONNECTION',
        (ChannelSftp.SSH_FX_CONNECTION_LOST): 'CONNECTION_LOST',
        (ChannelSftp.SSH_FX_OP_UNSUPPORTED): 'UNSUPPORTED'
    ]

    private String host
    private int port
    private String user
    private String password
    private String path

    public SftpHelper(Binding binding) {
        Properties scriptParameters = binding.variables.scriptParameters
        host = scriptParameters['sftp.host']
        port = scriptParameters['sftp.port'] as int
        user = scriptParameters['sftp.user']
        password = scriptParameters['sftp.password']
        path = scriptParameters['sftp.path']
    }

    public String download(String fileName, File outFile) {
        def jsch = new JSch()
        def session = jsch.getSession(user, host, port)
        session.setPassword(password);
        def config = new Properties();
        config["StrictHostKeyChecking"] = "no";
        session.setConfig(config);

        ChannelSftp channel = null;
        OutputStream out = null;
        try {
            out = new FileOutputStream(outFile)
            session.connect()
            channel = session.openChannel("sftp")
            channel.connect()
            channel.cd(path)
            channel.get(fileName, out)
            return "OK"
        } catch (SftpException e) {
            return CODE_TO_MSG[e.id]
        } finally {
            try {
                session.disconnect()
            } catch (Exception e) {
                /* Ignore */
            }
            try {
                channel.disconnect()
            } catch (Exception e) {
                /* Ignore */
            }
            try {
                out.close()
            } catch (Exception e) {
                /* Ignore */
            }
        }
    }
}

It uses the following parameters (adjust according to your server):

sftp.host = localhost
sftp.port = 22
sftp.user = username
sftp.password = password
sftp.path = /remote/server/root

file.name = remote-file.txt
file.local = /tmp/file.txt

And here is a usage of the library script:

def sftp = new SftpHelper(binding)
Map<String, String> scriptParameters = binding.scriptParameters
sftp.download(scriptParameters['file.name'],
        new File(scriptParameters['file.local']))

4.3.2. Reading and updating custom field values

Working directly with custom field values can be a hard task. For this reason, Cyclos provides the scriptHelper.wrap(object[, customValues]) method. It will return a Map<String, Object> which can be used to set and get custom field values. When setting a value, the input will be converted the value to the expected type, whereas when reading a value, the 'native' data type is returned. Here is an example:

def bean = scriptHelper.wrap(user)

// In this case we're using the internal name of the possible value
bean.gender = 'male'

// Here, gender is of type org.cyclos.entities.system.CustomFieldPossibleValue
def gender = bean.gender

// Here, date will be a java.util.Date
def date = bean.customDate

// Here, relatedUser will be an instance of org.cyclos.entities.users.User
def relatedUser = bean.relatedUser

Note that when you call wrap(object) without the second parameter, Cyclos will try to determine which are all the available custom fields. You can also pass in the second argument, which is a collection of org.cyclos.entities.system.CustomFields to specify exactly which are the fields to use.

Also, note that you don’t need to wrap every object (we’ve seen many clients doing this). You only should wrap objects when reading / writing custom values.

4.3.3. Custom locks

Sometimes it might be interesting to have system-wide locks, specially in a cluster. Cyclos implements locks using PostgreSQL’s advisory locks. All locks are held until the transaction ends (either committed or rolled back).

Cyclos provides the org.cyclos.impl.locks.LockHandler component to acquire locks. Starting with version 4.16, custom locks are supported. These locks are meant to be used by scripts.

For an example in a custom lock usage, refer to this example.

4.3.4. Reusing shared object instances between scripts

Sometimes it is desirable to have a shared object between all script executions in the same JVM. To accomplish this, a variable bound to all scripts named globals is available. It is a ConcurrentMap<String, Object>. Please, notice that ConcurrentMap doesn’t allow null s neither for keys nor values.

This map can be used on 2 situations:

  • For storing objects that should be cached and reused, such as external connection pools or service clients. In this case, scripts should use the Map.computeIfAbsent() method to get an existing or produce a new instance if none exists. When the Cyclos server is shutdown, all values in this map will be destroyed if they either follow a Spring destruction semantics (having a method annotated with @PreDestroy or implementing DisposableBean) or by implementing AutoCloseable;

  • For storing simple variables that are shared between the scripts. In this case, the recommended method to read and update the value is Map.compute(), to avoid concurrency issues. If many values will be added to the map, those should be removed to avoid memory leaks. The size of the map is shown in the system monitor page under 'Scripting global storage size'.

Consider these points:

  • There will probably be concurrent access to the values in the globals map, so only use instances that are thread-safe;

  • In a cluster, the globals map is local to each node. However, you can obtain the values in all nodes by calling clusterHandler.getGlobal('key'). For this to work, the values must be serializable. The resulting map is keyed by the host id of each node;

  • It is always advisable to use either Map.computeIfAbsent() or Map.compute() methods instead of directly accessing the map values. This will avoid headaches!

  • In development, if you need to recreate the value in the map, you can run the following in System > Tools > Run script: globals.remove('name').

Here are some examples:

Connecting to an external MariaDB / MySQL database

This examples provides a connection pool to an external MariaDB / MySQL database, which could be further accessed from other scripts.

@Grab(group='org.mariadb.jdbc', module='mariadb-java-client', version='3.0.8')

import javax.sql.DataSource

import org.springframework.jdbc.core.JdbcTemplate
import org.springframework.jdbc.datasource.DataSourceTransactionManager
import org.springframework.transaction.PlatformTransactionManager
import org.springframework.transaction.support.TransactionTemplate

import org.mariadb.jdbc.MariaDbPoolDataSource

class MariaDbConnection {
    DataSource dataSource
    PlatformTransactionManager transactionManager
    TransactionTemplate transaction
    JdbcTemplate jdbc

    MariaDbConnection(Binding binding) {
        Map<String, Object> globals = binding.variables.globals
        Properties scriptParameters = binding.variables.scriptParameters

        dataSource = globals.computeIfAbsent('mariaDb.pool') {
            new MariaDbPoolDataSource(scriptParameters.jdbcUrl)
        }
        transactionManager = globals.computeIfAbsent('mariaDb.txManager') {
            new DataSourceTransactionManager(dataSource)
        }
        transaction = globals.computeIfAbsent('mariaDb.txTemplate') {
            new TransactionTemplate(transactionManager)
        }
        jdbc = globals.computeIfAbsent('mariaDb.jdbc') {
            new JdbcTemplate(dataSource)
        }
    }
}

This example uses script parameters. Note that the JDBC URL has many parameters. Refer to https://mariadb.com/kb/en/pool-datasource-implementation/ for more details:

jdbcUrl = jdbc:mariadb://localhost:3306/dbname?user=dbuser&password=dbpwd&maxPoolSize=10

And this is an example of the usage from another script that uses the library:

def db = new MariaDbConnection(binding)

// Run a transaction
db.transaction.execute {
    def rows = db.jdbc.queryForList("select a, b from my_table")
    rows.each { println("A: ${it.a}, B: ${it.b}") }
}
Reusing a single HttpClient for all scripts

This examples allows reusing a single instance of HttpClient. This class creates some threads (which are stopped when the garbage collector runs), but can still impose some performance penalty if instantiated too many times. As the instance is thread-safe and can be reused, it is a good practice to do so. For simplicity, the example uses the map directly, but, just like in the previous example, it is advised to have the common code in a library script.

import java.net.http.HttpClient

HttpClient client = globals.computeIfAbsent('httpClient') {
    HttpClient.newBuilder()
            // Reuse Cyclos' shared executor
            .executor(invokerHandler.executorService)
            .build()
}
A simple counter

This example simply increments a value on every call. In a cluster, this counter will be per-node.

int counter = globals.compute('counter') { k, Integer value ->
    (value ?: 0) + 1
}

And the following example is slightly modified to provide a cluster-wide view of the counter:

// First lock, to avoid 2 cluster members to read / increment
lockHandler.lock('counter')

// Now return the maximum local counter value on each node
int counter = globals.compute('counter') { k, v ->
    def current = clusterHandler.getGlobal(k).values().max() ?: 0
    return current + 1
}

4.4. Script types

4.4.1. Library

Libraries are scripts which are included by other scripts, in order to reuse code, and are never used directly by other functionality in Cyclos.

Each script (including other libraries) can have any number of libraries as dependencies. However, circular dependencies between libraries (for example, A depends on B, which depends on C, which depends on A) are forbidden (validated when saving a library).

The order in which the code on libraries is included in the final code respects the dependencies, but doesn’t guarantee ordering between libraries in the same level. For example, if there are both C and B libraries which depend on A, it is guaranteed that A is included before B and C, but either B or C could be included right after A. So, in the example, your code shouldn’t rely on that B comes before C. In this case, the library C should depend on B to force the A, B, C order.

Contrary to other script types, libraries don’t have bound variables per se: the bindings will be the same as the script including the library.

Also, as libraries are just prepended in other scripts, no direct examples are provided here. The example scripting solutions, however, all use libraries.

4.4.2. Custom field validation

These scripts are used to perform custom validation logic to custom field values. The field can be of any type (users, advertisements, user records, transactions and so on).

Additional bound variables

The script code has the following bound variables (besides the default bindings)

Note that in case of user custom field validation, the object may not be a org.cyclos.model.users.users.UserDTO. For instance, when paying an external user, object is a org.cyclos.model.banking.transactions.PerformExternalPaymentDTO.

Script result

The script should return one of the following:

  • A boolean: indicates that the value is either valid / invalid. When invalid, the general <Field name> is invalid error will be displayed;

  • A string: means the field is invalid, and the string is the error message. To concatenate the field name, use the {0} placeholder, like: {0} has an unexpected value;

  • Any other result will be considered valid.

Examples
E-mail

To have a custom field which is validated as an e-mail, use the following script:

import org.apache.commons.validator.routines.EmailValidator

return EmailValidator.getInstance().isValid(value)
IBAN account number

To validate an IBAN account number as a custom field, the following script can be used:

import org.apache.commons.validator.routines.checkdigit.IBANCheckDigit

return IBANCheckDigit.IBAN_CHECK_DIGIT.isValid(value.replaceAll("\\s", ""))
CPF Validation

In Brazil, people are identified by a number called CPF (Cadastro de Pessoas Físicas). It has 2 verifying digits, which have a known formula to calculate. Here’s the example for validating it in Cyclos:

import static java.lang.Integer.parseInt

def boolean validateCPF(String cpf) {
    // Strip non-numeric chars
    cpf = cpf.replaceAll("[^0-9]", "")

    // Obvious checks: needs to be 11 digits, and not all be the same digit
    if (cpf.length() != 11 || cpf.toSet().size() == 1) {
        return false
    }

    int add = 0
    // Check for verifier digit 1
    for (int i = 0; i < 9; i++) add += parseInt(cpf[i]) * (10 - i)
    int rev = 11 - (add % 11)
    if (rev == 10 || rev == 11) rev = 0
    if (rev != parseInt(cpf[9])) return false

    add = 0;
    // Check for verifier digit 2
    for (int i = 0; i < 10; i++) add += parseInt(cpf[i]) * (11 - i)
    rev = 11 - (add % 11)
    if (rev == 10 || rev == 11) rev = 0
    if (rev != parseInt(cpf[10])) return false

    return true
}

return validateCPF(value)

4.4.3. Load custom field values

These scripts are used to load a list of allowed values for a custom field. Custom fields of type dynamic selection are required to have such a script. Several other field types can have an optional load values script: string, integer, decimal, date, url, enumerated or linked entity. Enumerated fields naturally have a list of static possible values. The script, however, can be used to show a subset of those options to specific users. Multi-line text, rich text, boolean, image and file types cannot have a load custom field values script.

If a custom field of type string, integer, decimal, date, url or linked entity has a load values script, Cyclos will use a single selection or radio button group widget instead of the regular widget for the custom field. Also, when a load custom field values script is used, the server-side validation will ensure that saved values are valid according to the allowed values list.

The script has a separated code block which loads values for custom fields being used as a search filter. The field types supporting load values when filtering are: dynamic selection, linked entity and enumerated. In that case, the bound variables will be different from the ones for the code block that runs over fields used to create or edit some entity (user, advertisement, record, etc.).

Additional bound variables

In all cases, the script will have the following bound variables (besides the default bindings):

Also, depending on the custom field nature, there are the following additional bindings, both for the script that runs when creating or modifying an entity and for the script that runs over custom fields used as search filters:

User (profile) fields

When the field is used for registering a user or editing a user profile:

When the field is used as search filter or in the built-in bulk action, Change custom field value:

  • searchContext: The org.cyclos.impl.users.ProfileFieldSearchContext in which the custom field is being used for search. Note that only the values reflecting filters are used. This enumeration also contains cases for the field in keywords, but it will never be the case when calling this script. In the bulk action mentioned above, this parameter is null.

  • overBrokeredUsers: This flag indicates, in case a broker is logged in, if the search is being done only over users he/she manages (true) or if this is a general search, as member (false).

  • Also, as user custom profile fields can be used to search advertisements or records, the same variables bound for those custom fields when used as search filter will also be available for the user profile fields. See below for the extra variables bound for advertisement and record fields when used as search filters.

Contact fields (personal contact list)

When the field is used for creating or modifying a contact:

When the field is used for search the contact list:

Public contact information fields

When the field is used for creating or modifying a public contact information:

There is no public contact information search, so the secondary script code doesn’t apply.

Advertisement fields

When the field is used for creating or modifying an advertisement:

When the field is used for searching advertisements:

  • user: If searching advertisements of a specific user, contains the org.cyclos.entities.users.User instance;

  • adType: The type of advertisements being searched. It is null when all types are being searched. Otherwise, contains the org.cyclos.model.marketplace.advertisements.AdType instance;

  • overBrokeredUsers`: This flag indicates, in case a broker is logged in, if the search is being done only over managed users (true) or if this is a general advertisements search (false).

Record fields

When using the field for creating or modifying a record:

When the field is used for searching records:

  • recordType: In most cases, the record type is known, and this contains the org.cyclos.entities.users.RecordType. However, it is also possible to search for records using shared record fields, over multiple record types at the same time. In that case, this variable will be null.

Transaction fields

When the field is used to perform a payment:

When the field is used to search an account history:

Custom operation fields

When the field is used to run a custom operation:

Custom operation fields are never used on search, so the secondary script code doesn’t apply.

Dynamic document fields

When the document is being printed:

Dynamic document fields are never used on search, so the secondary script code doesn’t apply.

Custom wizards fields

When the field is being shown in a form fields step:

Custom wizards fields are never used on search, so the secondary script code doesn’t apply.

Voucher fields

When the field is being shown in a voucher creation or activation:

Voucher fields are never used on search, so the secondary script code doesn’t apply.

Script result

The expected result type should match the custom field type. Must be either one, a collection or an array of:

Dynamic selection
  • String: In this case, each allowed possible value will have both value and label equals.

  • org.cyclos.model.system.fields.DynamicFieldValueVO (or compatible object / Map): The dynamic field value, containing the properties value (the internal value) and a label (the display value). The value must be not blank, or an error will be raised. If the label is blank, it will show the same text as the value. Also, the first dynamic value with defaultValue set to true will show up by default in the form.

String

Any returned object will be converted to string via toString().

Numeric (integer or decimal)

The result may be either numbers or strings

Date
Linked user
Linked transaction
Linked transfer
Linked record
Linked advertisement
Examples
Dynamic selection on user profile field: values depending on the user group

This example applies to a custom user profile field, and returns distinct values according to the user group.

import org.cyclos.model.system.fields.DynamicFieldValueVO

def values = []
// Common values
values << new DynamicFieldValueVO("common1", "Common value 1")
values << new DynamicFieldValueVO("common2", "Common value 2")
values << new DynamicFieldValueVO("common3", "Common value 3")
if (user.group.internalName == "business") {
    // Values only available for businesses
    values << new DynamicFieldValueVO("business1", "Business value 1")
    values << new DynamicFieldValueVO("business2", "Business value 2")
    values << new DynamicFieldValueVO("business3", "Business value 3")
} else if (user.group.internalName == "consumer") {
    // Values only available for consumers
    values << new DynamicFieldValueVO("consumer1", "Consumer value 1")
    values << new DynamicFieldValueVO("consumer2", "Consumer value 2")
    values << new DynamicFieldValueVO("consumer3", "Consumer value 3")
}
return values

And here is the script returning all available values, to be used for search filters:

import org.cyclos.model.system.fields.DynamicFieldValueVO

return [
    new DynamicFieldValueVO("common1", "Common value 1"),
    new DynamicFieldValueVO("common2", "Common value 2"),
    new DynamicFieldValueVO("common3", "Common value 3"),
    new DynamicFieldValueVO("business1", "Business value 1"),
    new DynamicFieldValueVO("business2", "Business value 2"),
    new DynamicFieldValueVO("business3", "Business value 3"),
    new DynamicFieldValueVO("consumer1", "Consumer value 1"),
    new DynamicFieldValueVO("consumer2", "Consumer value 2"),
    new DynamicFieldValueVO("consumer3", "Consumer value 3")
]
Linked user: active brokers

This example applies to a custom field of type linked entity - user. It returns all active brokers in the system, so the user can select one.

import org.cyclos.model.access.Role
import org.cyclos.model.users.users.UserQuery
import org.cyclos.model.users.users.UserStatus

def q = new UserQuery()
q.setUnlimited()
q.ignoreProfileFieldsInList = true
q.roles = [Role.BROKER]
q.userStatus = [
    UserStatus.ACTIVE,
    UserStatus.BLOCKED
]
return userService.search(q)
Linked transaction on transaction field: list open loans

This example lists all transactions of a specific payment type (loan grant) to the user performing the payment, filtering by a specific transfer status (open). It could be used on a payment from user to system to repay the loan, which would also need additional processing from an extension point script to mark the loan as repaid (script not included in this example).

import org.cyclos.entities.banking.AccountType
import org.cyclos.model.banking.accounts.AccountHistoryQuery
import org.cyclos.model.banking.accounts.AccountVO
import org.cyclos.model.banking.transferstatus.TransferStatusVO
import org.cyclos.model.banking.transfertypes.TransferTypeVO

// Find the account
def accountType = entityManagerHandler.find(AccountType, 'user')
def account = accountService.load(fromOwner, accountType)

// The account history has transfers. We need the transactions.
def q = new AccountHistoryQuery()
q.setUnlimited()
q.account = new AccountVO(account.id)
q.transferTypes = [
    new TransferTypeVO(internalName: 'debit.loan')
]
q.statuses = [
    new TransferStatusVO(internalName: 'loan.open')
]
def transfers = accountService.searchAccountHistory(q).pageItems

// Return the transaction ids
return transfers.collect {it.transactionId}

4.4.4. Account number generation

This kind of script is responsible for generating account numbers, in case more control than the default (random generation) is needed.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script should return a string, which should match the account number mask set in the configuration (if any). If the string returns null or an empty string, no number is assigned to the account.

The script doesn’t need to check if the account number already exists. This is done internally. If the number is already used, the script is called again (up to 10 times, then, an error is raised).

Examples
Controlling the prefix according to the currency and user group

In this example, the mask ##-####### is expected for the account number. The prefix is composed of 2 digits:

  • The first one is 0 if the currency is unit, or 1 otherwise.

  • The second one is 0 for system, 1 for business, 2 for consumers of 9 otherwise.

The rest are 7 random digits.

import org.cyclos.entities.users.User
import org.cyclos.utils.StringHelper

// Either unit or euro
String prefix = type.currency.internalName == 'internal_units' ? '0' : '1'

if (owner instanceof User) {
    switch (owner.group.internalName) {
        case 'business':
            prefix += '1'
            break
        case 'consumers':
            prefix += '2'
            break
        default:
            prefix += '9'
    }
} else {
    prefix += '0'
}

return prefix + "-" + StringHelper.randomNumeric(7)

4.4.5. Account fee calculation

These scripts are used to calculate the amount of an account fee over a specific user. An account fee is charged periodically or manually over many accounts, according to the Charged account fees setting in member products.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script should return a number, which will be rounded to the currency’s decimal digits. If null or zero is returned, the fee is not charged, and the user is skipped.

Examples
Charge a different amount according to the user rank

This example allows choosing a distinct account fee amount based on a profile field of the paying user. It is assumed a custom field of type single selection with the internal name rank. It should have 3 possible values, with internal names 'bronze', silver and gold.

// Depending on a user custom field, we'll pick the fee amount
def amounts = [bronze: 10, silver: 7, gold: 5]
def user = scriptHelper.wrap(account.owner)
def rank = user.rank?.internalName ?: "bronze"
return amounts [rank]

4.4.6. Transfer fee calculation

These scripts are used to either calculate the amount or the full characteristics of a transfer fee (a fee triggered by another transfer).

Starting with Cyclos 4.16, the transfer fee details page has a field called 'Configuration mode', which can be set to either 'Form' or 'Script'.

  • Form: In this mode, all transfer fee characteristics are statically defined in the form. The only characteristic that can be defined by the script is the actual fee amount, when the 'Charge mode' field is set to 'Custom script'.

  • Script: In this mode, the script defines who pays, who receives, the generated payment type and the amount of the fee.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script result is interpreted according to the transfer fee’s configuration mode:

  • Form: The script should return a number, which will be rounded to the currency’s decimal digits. If null or zero is returned, the fee is not charged.

  • Script: The script should return an instance of org.cyclos.impl.banking.TransferFeeRecipe, or compatible object or Map. The script should set the following fields:

    • amount: The amount to be charged, as a number. When null or not positive, the fee is not charged;

    • from: The account owner that pays the fee. Can be one of:

    • to: The account owner that receives the fee. Same accepted values as from;

    • type: The generated transfer type. It must have been created as a generated transfer type, not payment transfer type. When represented as string, is the composite internal name, in the form accountTypeInternalName.transferTypeInternalName.

Examples
Charging a fee according to a user profile field

This example allows choosing a distinct fee amount based on a profile field of the paying user. The fee must have been configured using the form. It is assumed a custom field of type single selection with the internal name rank. It should have 3 possible values, with internal names bronze, silver and gold. The script then chooses a different percentage according to the user rank.

if (transfer.fromSystem) {
    // Only charge users
    return 0
}

// Depending on a user custom field, we'll pick the fee amount
def percentages = [bronze: 0.07, silver: 0.05, gold: 0.02]
def from = scriptHelper.wrap(transfer.fromOwner)
def rank = from.rank?.internalName ?: "bronze"
def percentage = percentages[rank]
return transfer.amount * percentage
Charging a fee according to a payment custom field

This example is similar to the above, but based on a transaction custom field in the payment itself. The main difference is the source for custom field values now depends on whether we’re calculating the fee during a payment preview (used to show the user the paid fees before the transfer is actually processed) or for the actual transfer processing. That is because the transfer.transaction is not available during preview. The fee must have been configured using the form.

However, to allow retrieving the custom fields during preview, there is an extra bound variable, called previewParameters (not available during transfer processing). Similar to the previous example, but this one assumes the single selection field has internal name category, and the possible values have internal names loan, repayment and buying.

def percentages = [loan: 0.05, repayment: 0.01, buying: 0.02]
def source = previewParameters ?: transfer.transaction
def bean = scriptHelper.wrap(source)
def category = bean.category?.internalName ?: "buying"
def percentage = percentages[category]
return transfer.amount * percentage
Distributing a fee exactly to different accounts

Some systems charge a fee from users (be it a transfer fee or account fee) which is itself distributed amongst different accounts. For example, a 5% transaction fee is charged from users, and that fee amount is distributed like 12% to account A, 27% to account B and 61% to account C. So, the transaction fee transfer type itself has other 3 fees.

The problem in making them all percentage is that each fee charge rounds the charged amount (generally to 2 decimal places, according to the currency), and that may cause the total distributed amount to be different from the total fee amount.

A solution for this problem is to make one of the fees calculated by script, so it sums up what each other fee has charged, and charges the remaining. Generally, the fee with the largest charge percentage would then use this script, while all other fees will be configured as percentages.

The fee must have been configured using the form.

import org.cyclos.entities.banking.Transfer
import org.cyclos.model.banking.transferfees.TransferFeeChargeMode
import org.cyclos.utils.BigDecimalHelper

Transfer transfer = binding.transfer
BigDecimal amount = transfer.amount

// Sum what the other fees will charge
int scale = transfer.currency.precision
BigDecimal others = 0
for (def fee in transfer.type.transferFees) {
    if (fee.chargeMode == TransferFeeChargeMode.PERCENTAGE) {
        others += BigDecimalHelper.round(amount * fee.amount, scale)
    }
}

// Charge the rest
return BigDecimalHelper.round(amount - others, scale)
Pay a transfer fee, either to a broker or system

In this example, the user will pay a 1% transfer fee either to his main broker, if the user has a broker, or otherwise will pay to a system account.

This cannot be achieved when the fee is configured by form. Instead, it must be configured using script, because in the example, both the receiver and the generated type are dynamic.

import org.cyclos.entities.banking.Transfer
import org.cyclos.entities.users.User

Transfer transfer = binding.transfer
BigDecimal amount = transfer.amount

def fromUser = transfer.fromOwner as User
return [
    amount: amount * 0.01,
    from: fromUser,
    to: fromUser.mainBroker ?: 'system',
    type: fromUser.mainBroker ? 'units.feeToBroker' : 'units.feeToSystem'
]

4.4.7. Transfer status handling

These scripts are used to determine to which status(es) a transfer may be set after the current status. By default, if no script is used, the possible next statuses (as configured in the transfer status details page) will be available.

Using a script, however, allows using finer-grained controls. For example, a specific status could be allowed only by specific administrators, or only under special conditions (for example, checking the account balance or any other condition).

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script must return one of the following:

  • A single / collection / array / iterator of org.cyclos.entities.banking.TransferStatus: The possible next statuses;

  • A single / collection / array / iterator of strings: The internal names of the possible next statuses;

  • A single / collection / array / iterator of numbers: The internal id (same as the database id, not the masked id returned to clients) of the possible next statuses;

  • Null: Assumes the default behavior - the possible next configured in the status are used.

Examples
Restricting a specific status for administrators

In this example, any user can change a transfer status in a given flow. However, only administrators can set a transfer to the status with internal name finished.

// Only administrators can set the status to finished
return status.possibleNext.findAll { st ->
    sessionData.admin || st.internalName != "finished"
}

4.4.8. Session handling

These scripts can be used to manage user sessions (logins) externally. It can only be set in the network default configuration, as the custom session handling script. There are distinct code blocks in this script type, each implementing a different aspect.

On any of these functions, returning null or having an empty code block will result in the default session management taking place. This way, it is possible to implement a custom handling only on special cases. For example, a custom session mechanism might be used only for privileged administrators, whose session tokens comply with a specific format.

For reference, Cyclos sessions use 32-character alphanumeric strings, with no punctuation. So, for example, if session tokens generated for those administrators have a different format, say, a UUID, the script can differentiate which sessions tokens correspond to normal sessions (and return null on the Logout and Resolve functions for those token format) and handle only those specific sessions. Also, in such a case, the login method could check the user group being logged in, and either perform the login on the underlying system (returning the generated session token) or return null for regular users.

Caution: Errors on any of these functions, specially the first three, may cause users not being able to login or access the system. A good security measure while developing such scripts is to handle a specific (for example, if the login name is 'admin') with the default session resolution, and withdraw this case after the rest of the script is ready. If such a situation occurs, a possible workaround is to login in global mode, then disable and lock the custom session handling in the configuration from which the network configuration inherits. Then edit the script and unlock it again in global mode.

Login

This script function is called when a session is being created.

Additional bound variables

The login function has the following bound variables (besides the default bindings):

Script result
  • String: A session token. All other session properties will be handled as default;

  • Null: When returning null, the default session handling will be used for this user.

Logout

This script function is called when a session is terminated.

Additional bound variables

The logout function has the following bound variables (besides the default bindings):

  • sessionToken: The session token for the session being invalidated;

  • remoteAddress: The remote IP address (string) over which the session is being invalidated.

Script result
  • True: If the script returns true, Cyclos considers the logout handled by the script;

  • False or Null: In this case, the default session handling will be used for invalidating this session.

Resolve

This script function is called on every request, to resolve which user belongs to a session token.

Additional bound variables

The resolve function has the following bound variables (besides the default bindings):

  • sessionToken: The input session token;

  • remoteAddress: The remote IP address (string) over which the session is being resolved.

Script result
Set properties

This script function is called when specific session properties are modified. Most likely, this will be called when login confirmation is enabled with a trusted device. In that case, the previously untrusted session will be considered trusted.

Additional bound variables

The set properties function has the following bound variables (besides the default bindings):

Script result

The set properties function result is ignored.

This script is called when an administrator searches for connected users, as well as on the administrator home page, as the number of connected users is shown.

Additional bound variables

The search connected users function has the following bound variables (besides the default bindings):

Script result
Examples
Storing sessions on Cyclos script storages

This example stores user sessions in the scripting storage. It is not a realistic example, as Cyclos itself is used to store sessions, but it does demonstrate the usage of a session handling script. Here are the sources for each of the 5 code boxes:

Function to perform the login:

import org.apache.commons.lang3.RandomStringUtils
import org.cyclos.entities.access.Channel
import org.cyclos.entities.access.SessionProperties
import org.cyclos.entities.users.UserPrincipal
import org.cyclos.entities.utils.TimeInterval
import org.cyclos.impl.system.ScriptStorageHandler

ScriptStorageHandler scriptStorageHandler = binding.scriptStorageHandler
UserPrincipal principal = binding.principal
TimeInterval sessionTimeout = binding.sessionTimeout;
String remoteAddress = binding.remoteAddress
SessionProperties sessionProperties = binding.sessionProperties
Channel channel = binding.channel

String token = RandomStringUtils.randomAlphanumeric(64)
int timeout = sessionTimeout.milliseconds / 1000
def storage = scriptStorageHandler.get("session_${token}", timeout)
storage.principal = principal
storage.remoteAddress = remoteAddress
storage.timeout = sessionTimeout
storage.sessionProperties = sessionProperties
storage.channel = channel

return token

Function to perform the logout:

import org.cyclos.impl.system.ScriptStorageHandler

ScriptStorageHandler scriptStorageHandler = binding.scriptStorageHandler
String sessionToken = binding.sessionToken

return scriptStorageHandler.remove("session_${sessionToken}")

Function to resolve a session given a token:

import org.cyclos.entities.access.Session
import org.cyclos.impl.system.ScriptStorageHandler

ScriptStorageHandler scriptStorageHandler = binding.scriptStorageHandler
String sessionToken = binding.sessionToken

def storage = scriptStorageHandler.getIfValid("session_${sessionToken}")
Session session = null
if (storage != null) {
    session = new Session()
    session.initFrom(storage.principal)
    session.sessionToken = sessionToken
    session.properties = storage.sessionProperties
    session.channel = storage.channel
    session.remoteAddress = storage.remoteAddress
    session.sessionTimeout = storage.timeout
}
return session

Function to set the session properties:

import org.apache.commons.lang3.RandomStringUtils
import org.cyclos.entities.access.SessionProperties
import org.cyclos.entities.utils.TimeInterval
import org.cyclos.impl.system.ScriptStorageHandler

import org.cyclos.entities.users.BasicUser
ScriptStorageHandler scriptStorageHandler = binding.scriptStorageHandler
String sessionToken = binding.sessionToken
SessionProperties sessionProperties = binding.sessionProperties

def storage = scriptStorageHandler.getIfValid("session_${sessionToken}")
if (storage != null) {
    storage.sessionProperties = sessionProperties
}

Function to search for connected users:

import org.cyclos.entities.system.QScriptStorage
import org.cyclos.entities.system.ScriptStorage
import org.cyclos.entities.users.UserPrincipal
import org.cyclos.impl.utils.persistence.EntityManagerHandler
import org.cyclos.model.users.users.ConnectedUserQuery
import org.cyclos.server.utils.JacksonParameterStorage
import org.cyclos.utils.PageImpl
import org.cyclos.utils.StringHelper

import com.fasterxml.jackson.databind.ObjectMapper

ConnectedUserQuery query = binding.query
EntityManagerHandler entityManagerHandler = binding.entityManagerHandler
ObjectMapper objectMapper = binding.objectMapper
QScriptStorage ss = QScriptStorage.scriptStorage

// First we get the persisted script storages which start with 'session_'
PageImpl page = entityManagerHandler
        .from(ss)
        .where(ss.key.like("session\\_${StringHelper.repeat('_', 64)}", '\\'.charAt(0)),
        ss.expirationDate.after(new Date()))
        .orderBy(ss.creationDate.asc())
        .page(query, ss)

// Each one is parsed as JSON and converted to the expected format
page.pageItems = page.pageItems.collect { ScriptStorage it ->
    def storage = new JacksonParameterStorage(objectMapper, it.content)
    UserPrincipal principal = storage.principal
    return [
        sessionToken: StringHelper.removeStart(it.key, "session_"),
        user: principal.basicUser,
        creationDate: it.creationDate,
        channel: storage.channel,
        remoteAddress: storage.remoteAddress
    ]
}
return page

4.4.9. Password handling

These scripts are used to check passwords. In order to use them, the password type’s password mode needs to be Script.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script should return a boolean, indicating whether the password is ok or not.

Examples
Matching passwords to the script parameters

This is a very simple example, which checks for passwords according to the script parameters. The parameters can be set either in the script itself or in the password type. This example is very insecure, and shouldn’t be used in production. Normally, scripts to check passwords would connect to third party applications, but this is just a very basic example.

// Just read the password value from the script parameters
return scriptParameters[user.username] == password

4.4.10. Extension points

These scripts are used on extension points (user, user record, transfer, …​), and are attached to specific events (create, update, remove, charge back, …​). The extension point scripts have 2 functions:

  • The data has already been validated, but not saved yet. In this function, we know that the data entered by users is valid, but the main event has not been saved yet;

  • The data has been saved, but not committed to database yet. In this function, the main event has been saved. For example, when registering a user, the user will be saved (and has an assigned id).

Here are some example scenarios for performing custom logic, or integrating Cyclos with external systems using extension points:

  • Custom credit limit. When a user is performing a payment, an extension point of type transaction could be used, in the function invoked after validation, to check the current balance. If the balance is not enough for the payment and the user has credit limit, a payment from a system account could be done automatically to the user, completing the amount for the payment;

  • A XA transaction could be done with an external system by creating data in the external database in the function which runs after validating, then preparing the commit in the function after the data is saved, and finally registering both a commit and a rollback listener (see the scriptHelper in default bindings) to either commit or rollback the prepared transaction;

  • A simple notification of performed payments could be implemented by registering a commit listener (see the scriptHelper in default bindings) to implement the notification;

  • The profile information of a user needs to be mirrored in an external system. In this case, a user extension point, with the create / update events, can be used to send this information. Additional information on addresses and phones can use the same mechanism (they are different extension points);

  • There could be payment custom fields which are not filled-in by users when performing payments, but by extension points of type transaction. Payment custom fields may be configured to not show up in the form, only automatically via extension points;

  • An extension point on a new Cyclos advertisement could publish the advertisement as well in a third party system.

These are just some examples. There are many possible uses for the extension points.

Additional bindings

All extension points have the following bound variables (besides the default bindings):

The following types of extension points exist:

User extension point

Extension points which monitor events on users, including administrators, brokers and regular users.

Additional bindings
Events
  • Create: a user is being registered. IMPORTANT: When e-mail validation is enabled, the user will be pending until confirming the e-mail. If you have e-mail confirmation enabled, this event might not be what you need, but activate instead;

  • Activate: a user is being activated for the first time. If enabling email validation, after the user confirms the email address, this event will be triggered. When email validation is not enabled, users will be initially active if the group’s Initial status for users field is active. Otherwise, only when the user is manually activated, this event will be triggered;

  • Update: The user profile fields are being updated. Additional bindings:

  • Change group: The user’s group is being changed. Additional bindings:

  • Change status: The user status is being changed. Additional bindings:

Operator extension point

Extension points which monitor events on operators.

Additional bindings
Events
Address extension point

Extension points which monitor events on user addresses.

Additional bindings
Events
  • Create: An address is being created;

  • Update: An address is being updated. Additional bindings:

  • Delete: An address is being deleted.

Phone extension point

Extension points which monitor events on user phones.

Additional bindings
Events
  • Create: A phone is being created;

  • Update: A phone is being updated. Additional bindings:

  • Delete: A phone is being deleted.

Record extension point

Extension points which monitor events on records.

Additional bindings
Events
  • Create: A record is being created;

  • Update: A record is being updated. Additional bindings:

  • Delete: A record is being deleted.

Advertisement extension point

Extension points which monitor events on advertisements.

Additional bindings
Events
  • Create: An advertisement is being created;

  • Update: An advertisement is being updated. Additional bindings:

  • Delete: An advertisement is being deleted.

Transaction extension point

Extension points which monitor events on transactions.

Additional bindings

The following additional bindings are available for both the Preview, Confirm, Send payment request and Create ticket events:

Events
Transaction authorization extension point

Extension points which monitor events on a transactions.

Additional bindings
Events
  • Authorize: The transaction is being authorized. Be careful: there might be more authorization levels which need to be authorized before the transaction is finally processed. Additional bindings:

  • Deny: The transaction is being denied by the authorizer;

  • Cancel: The transaction is being canceled by the performer;

  • Expire: The transaction is being expired by the system through a polling task. If the transfer type requires authorization, it is possible to define an expiration period to avoid leaving the payment indefinitely in a pending state.

Transfer extension point

Extension points which monitor events on a transfer.

Additional bindings
Events
Voucher extension point

Extension points which monitor events on vouchers.

Additional bindings
Events
  • Generate: A voucher is being generated;

  • Buy: A voucher is being bought or sent by a user. To differentiate, use the flag voucher.pack.sent;

  • unblock: A blocked voucher is being unblocked;

  • redeem: A voucher is being redeemed. Additional bindings:

    • user or redeemer: The voucher redeemer (generally a shop or cash point) as org.cyclos.entities.users.User;

    • amount: The amount being redeemed. May be less than the voucher amount if the type allows partial redeems.

  • Cancel: A generated voucher is being canceled. Additional bindings:

  • Expire: A voucher has expired. Additional bindings:

Agreement extension point

Extension points which monitor events on user agreements.

Additional bindings
Events
  • Accept: An agreement is being accepted;

  • Reject: An optional agreement which was previously accepted is no longer accepted.

Import extension point

Extension points which monitor events on imports.

Additional bindings
Events
  • File status changed: The status of the imported file has changed. Additional bindings:

  • Line read: An imported line was read from the CSV file. Additional bindings:

  • Line processed: A line is being processed. On the validated phase, the line isn’t yet processed. On the saved phase, the line was processed, either with success or error. Additional bindings:

    • line: The org.cyclos.entities.system.ImportedLine being processed;

    • entity: Only on the saved phase when success (null when error). The entity which was created. The actual type depends on the import type. Can be a user, an advertisement, a transfer, a transaction, a record, a voucher, etc.;

    • error: Only on the saved phase when error (null when success). The Java error which was thrown when processing the line.

Examples
Granting extra credit (on demand) before payments

This example allows, with a custom profile field, to define an extra credit limit the user can use on demand. When performing a payment, if the available balance is not enough, a payment is performed from a system account to the user, up to the limit specified in that profile field. Once the payment is done, the profile field is subtracted

This example expects the system account to have the internal name debitUnits, and it should have a payment transfer type to the user account. That payment transfer type should have the internal name extraCredit. Finally, the custom profile field needs to have the internal name availableCredit, and needs to be of type decimal, and enabled for the user.

Then create an extension point of type Transaction, enabled and for the confirm event. This example only works for payments without fees. Use this in the "Script code executed when the data is saved" code block:

import org.cyclos.entities.banking.Account
import org.cyclos.entities.banking.PaymentTransferType
import org.cyclos.entities.banking.SystemAccountType
import org.cyclos.model.banking.accounts.SystemAccountOwner
import org.cyclos.model.banking.transactions.PerformPaymentDTO
import org.cyclos.model.banking.transfertypes.TransferTypeVO

// Only process direct payments. Scheduled payments are skipped
if (!(performTransaction instanceof PerformPaymentDTO)) {
    return
}

// Get the available credit as a profile field
def payer = scriptHelper.wrap(fromOwner)
BigDecimal availableCredit = payer.availableCredit?.abs()
if (availableCredit == null || availableCredit < 0.01) {
    // Nothing to do - no available credit
    return
}

// Get the account and balance
Account account = accountService.load(fromOwner, paymentType.from)
BigDecimal availableBalance = accountService.getAvailableBalance(account, null)
BigDecimal needs = performTransaction.amount - availableBalance
if (needs > 0 && needs <= availableCredit) {
    // Needs some extra credit, and has it available - make a payment from system
    // Find the system account and payment type
    SystemAccountType systemAccountType = entityManagerHandler.find(
            SystemAccountType, "debitUnits")
    PaymentTransferType paymentType =  entityManagerHandler.find(
            PaymentTransferType, "extraCredit", systemAccountType)
    PerformPaymentDTO credit = new PerformPaymentDTO()
    credit.owner = SystemAccountOwner.instance()
    credit.subject = fromOwner
    credit.type = new TransferTypeVO(paymentType.id)
    credit.amount = needs
    paymentService.perform(credit)
    // Now there should be enough credit to perform the payment

    // Update the user available credit
    payer.availableCredit -= needs
}
Send an e-mail on every payment

This example allows, for the selected payment types in the extension point details, to send an e-mail to a specific address. Use this in the "Script code executed when the data is saved" code block:

import javax.mail.internet.InternetAddress

import org.cyclos.model.ValidationException
import org.cyclos.server.utils.MessageProcessingHelper
import org.springframework.mail.javamail.MimeMessageHelper

// Get the e-mail subject and body
def tx = scriptHelper.wrap(transaction)
def vars = [
    payer: tx.fromOwner.name,
    amount: formatter.format(tx.currencyAmount),
    date: formatter.formatAsDate(new Date()),
    time: formatter.formatAsTime(new Date())
]
def subject = MessageProcessingHelper.processVariables(scriptParameters.subject, vars)
if (subject == null || subject.empty) {
    throw new ValidationException("Missing the 'subject' script parameter")
}
def body = MessageProcessingHelper.processVariables(scriptParameters.message, vars)
if (body == null || body.empty) {
    throw new ValidationException("Missing the 'message' script parameter")
}
def toEmail = tx.email
def fromEmail = sessionData.configuration.smtpConfiguration.fromAddress
def sender = mailHandler.mailSender

// Send the message after commit, so we guarantee the transaction is persisted
// when the e-mail is sent
scriptHelper.addOnCommit {
    def message = sender.createMimeMessage()
    def helper = new MimeMessageHelper(message)
    helper.to = new InternetAddress(toEmail)
    helper.from = new InternetAddress(fromEmail)
    helper.subject = subject
    helper.text = body
    // Send the message
    sender.send message
}
Assign / unassign individual products when the user accepts / rejects agreements

Starting with Cyclos 4.13, there are optional agreements. This example assigns / unassigns individual products to the user that accepts / rejects agreements.

The script parameters must be in the form:

agreementInternalName1=productInternalName1
agreementInternalName2=productInternalName2
...

Make sure the "Run with all permissions" checkbox is selected. Then, use this script in the "Script code executed when the data is saved" code block:

import org.cyclos.entities.access.Agreement
import org.cyclos.entities.users.User
import org.cyclos.impl.users.ProductsUserServiceLocal
import org.cyclos.model.system.extensionpoints.AgreementExtensionPointEvent
import org.cyclos.model.users.products.ProductVO
import org.cyclos.model.users.users.UserVO

// Get the variables from context
AgreementExtensionPointEvent event = binding.event
User user = binding.user
Agreement agreement = binding.agreement
ProductsUserServiceLocal productsUserService = binding.productsUserService
Map<String, String> scriptParameters = binding.scriptParameters

// Lookup the product by agreement internal name
def productVO = new ProductVO(internalName: scriptParameters[agreement.internalName])
def userVO = new UserVO(user.id)
def assigned = user.products.find { it.internalName == productVO.internalName } != null

if (event == AgreementExtensionPointEvent.ACCEPT) {
    // Assign the individual product
    if (!assigned) {
        productsUserService.assign(productVO, userVO)
    }
} else {
    // Unasign the individual product
    if (assigned) {
        productsUserService.unassign(productVO, userVO)
    }
}

In the extension point itself, select all agreements whose internal names are included in the script parameters, the user groups and both events.

Enforcing the user remains with a minimum balance for a payment type

This example forces the user to remain with a minimum balance for the payment types configured in the extension point. The extension point should be of type transaction, and the event should be confirmed. Paste the following on "Script code executed when the data is validated" code block:

import org.cyclos.entities.banking.Account
import org.cyclos.entities.utils.CurrencyAmount
import org.cyclos.impl.banking.AccountServiceLocal
import org.cyclos.impl.utils.formatting.FormatterImpl
import org.cyclos.model.ValidationException
import org.cyclos.model.banking.transactions.PerformTransactionDTO

Account account = binding.fromAccount
PerformTransactionDTO performTransaction = binding.performTransaction
AccountServiceLocal accountService = binding.accountService
Map<String, String> scriptParameters = binding.scriptParameters
FormatterImpl formatter = binding.formatter

def minBalance = new BigDecimal(scriptParameters.minBalance)

def balance = accountService.getBalance(account, null)
def newBalance = balance - performTransaction.amount
if (newBalance < minBalance) {
    throw new ValidationException("""This operation cannot be processed,
        as your new account balance would be
        ${formatter.format(new CurrencyAmount(account.currency, newBalance))},
        below the minimum allowed balance of
        ${formatter.format(new CurrencyAmount(account.currency, minBalance))}""")
}

4.4.11. Custom operations

These scripts are invoked when a user runs a custom operation. A custom operation is configured to return different data types, and the script must behave accordingly (see System – Operations for more details).

Custom operations can have different scopes:

  • System: Those are executed by administrators (with granted permissions), directly from the main menu;

  • User: Custom operations which are related to a user, and can either be executed by the own user (with granted permissions), from the main menu or run by administrator or brokers (also, with granted permissions) when viewing the user profile. In both cases, the custom operation needs to be enabled to users via member products. For example, there might be operations which apply only to businesses, not consumers, and even administrators with permission to run them shouldn’t be able to run them over consumers. It is enforced that administrators / brokers will only be able to run custom operations over users they manage;

  • Menu: These custom operations are executed by a custom menu entry. This is the only possible custom operation scope that can be run by guests. A classical example of this is a "Contact us" page;

  • Internal: An internal custom operation is executed either as an action (see below) or when the user clicks a row returned by another custom operation, which returns a table with results;

  • Advertisement: Custom operations which are executed over an advertisement;

  • Record: Custom operations which are executed over a record;

  • Transfer: Custom operations which are executed over a transfer (balance transfer between accounts);

  • Contact: Custom operations which are executed over a contact in a user’s contact list;

  • Public contact information: Custom operations which are executed over a public contact information in a user’s profile;

  • Bulk action: Custom operations executed on bulk actions, over each user individually.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

  • customOperation: The org.cyclos.entities.system.CustomOperation being executed;

  • user: The org.cyclos.entities.users.User. Only present if the custom operation’s scope is either user or bulk action;

  • bulkAction: The org.cyclos.entities.users.CustomOperationBulkAction. Only present if the custom operation’s scope is bulk action;

  • ad: The org.cyclos.entities.marketplace.BasicAd. Only present if the custom operation’s scope is advertisement;

  • record: The org.cyclos.entities.users.Record. Only present if the custom operation’s scope is record;

  • transfer: The org.cyclos.entities.banking.Transfer. Only present if the custom operation’s scope is transfer;

  • contact: The org.cyclos.entities.users.Contact. Only present if the custom operation’s scope is contact;

  • contactInfo: The org.cyclos.entities.users.ContactInfo. Only present if the custom operation’s scope is public contact information;

  • menuItem: The org.cyclos.entities.contentmanagement.MenuItem. Only present if the custom operation’s scope is menu;

  • inputFile: The org.cyclos.model.utils.FileInfo. Only present if the custom operation is configured to accept a file upload, and if a file was selected upon the operation execution;

  • formParameters: A Map<String, Object>, keyed by the form field internal name. The value depends on the custom field type (see the value binding on custom field validation script for details on types);

  • scannedQrCode: The string value scanned from the QR-code when submitting the operation. Only for custom operations which have 'Submit with a QR-code scan' set to true;

  • exportFormat: The org.cyclos.entities.system.ExportFormat indicating if an operation which returns a result page will return just the data (when null) or will be exported to a file;

  • currentPage: An integer indicating the current page, when getting paged results. Starts with zero. Only available if the result type is result page;

  • pageSize: An integer indicating the requested page size when getting paged results. Only available if the result type is result page;

  • skipTotalCount: A boolean indicating whether the total count should be skipped for this search. Only available if the result type is result page;

  • returnUrl: Only if the custom operation return type is external redirect. Contains the url (as string) which Cyclos expects the external site to redirect the user after the operation completes;

  • storage or parameterStorage: Only if the custom operation return type is external redirect. Contains an org.cyclos.server.utils.ObjectParameterStorage which is shared in both the first script and the callback handling script. This object is enhanced with propertyMissing methods, to support "syntactic sugar" on Groovy scripts, like storage.name = value. When this form is used, it is assumed that the input / output are plain strings;

  • execution or externalRedirectExecution: execution / externalRedirectExecution: Only if the custom operation return type is external redirect. Contains the org.cyclos.entities.system.ExternalRedirectExecution which stores the context for this execution;

  • request: The org.cyclos.model.utils.RequestInfo. Only if the custom operation return type is external redirect. Contains the information about the current request, so the script function which handles the callback can identify the context to complete the process.

Script result

The required return value depends on the custom operation result type. In all cases, the result type for the CustomOperationService.run() method is a org.cyclos.model.system.operations.RunCustomOperationResult. But, depending on the custom operation result type, the value returned by the script is handled differently, as shown below:

Plain text or Rich text

In these cases, the result has a title and a content. The script must return one of the following:

  • String: Is the result content. The header will be the custom operation name;

  • An object or Map containing the following properties:

    • content: The required result content;

    • title: The result title. When not specified, will use the operation name;

    • receipt: A receipt to be printed by the Cyclos mobile app in a Bluetooth printer. See below for more information on receipts.

Notification

In all cases, notifications are assumed to be HTML formatted. The script must return one of the following:

  • String: A plain string, which is considered as an information notification. If it is prefixed with either [INFO], [WARN] or [ERROR], such prefixes are removed from the notification and the notification level is set;

  • An object or Map with the following properties:

File download

The script must return an instance of org.cyclos.model.utils.FileInfo, or an object or Map with the same properties. The properties are:

  • content: Required. The file content. It may be an InputStream, a File or a String (containing the file content itself);

  • contentType: Required. The MIME type, such as text/plain, text/html, image/jpeg, application/pdf, etc.;

  • name: Optional file name, which will be used by browsers to suggest the file name to save;

  • length: Optional file length, which browsers use to monitor the progress of file downloads.

Page results

The script must return an object (or map) with the following properties:

  • columns: Either this or headers must be returned. Contains each column definition. Each column is a org.cyclos.model.system.operations.PageResultColumn or equivalent object. Each column can define a result property to display (otherwise it is assumed that each result is an array, accessed by index). Additionally, defines the type, header, width, align, valign. The type is a org.cyclos.model.system.operations.PageResultColumnType or equivalent string, such as: string, boolean, number, date or currencyAmount. Returning as currency amount is a special case, where exports can use this information to correctly format the amount. Also, when the type is boolean, number or date, results are sent with a suitable representation in a standard form, rather than a formatted string;

  • rows: A list of objects, each containing properties. Each column matches the corresponding object property to display each cell. An object can have additional properties, which can be used to pass parameters to the url when clicking a row;

  • Alternatively, and for backwards compatibility, instead of returning columns and rows, it is possible to return headers as a List<String> and results as a List<List<String>>, so the result table is assembled with those tabular data;

  • totalCount: Optional, used to page results. If a total count is returned, a result page navigator is shown to the user, and records can be returned page-by-page. The script should probably use the currentPage and pageSize variables to calculate the correct page to be returned;

  • hasNextPage: Indicates that there are more rows to be returned than this page. Ignored if totalCount is returned. Another way to enable pagination is to not return this flag explicitly, but to limit rows to pageSize + 1. When more results are returned than the page size, the list is truncated, but Cyclos has the information that there’s more data. Take into account that if both totalCount and hasNextPage are not returned and the rows list has exactly the same size as pageSize, then the pagination will be disabled because Cyclos will infer there is no more data.

URL

The script must return one of the following:

  • String: The URL, and the user is redirected to that URL in the same browser window;

  • An object or Map with the following properties:

    • url: The destination URL;

    • newWindow: A boolean value indicating whether the application will open a new browser window with the destination URL. Most browsers block popups by default, and opening in a new window is probably considered a popup by browsers. Hence, when opening a new window, on the first execution, users might be prompted whether the popup is allowed. Then they might need to run the operation again once the popup is allowed.

External redirect

This return type has 2 different scripts:

  • The first script should prepare the data in some external system, and then return the URL to which the user should be redirected. An example using this kind of script is the PayPal Integration, in which the first script creates a payment in PayPal, to be confirmed later on by the user. Two noteworthy variables bound to the script context which are necessary for this script are:

    • returnUrl: This is the URL that should be passed to the external service to redirect the user back to Cyclos to complete the operation;

    • storage: A storage which can be used to persist data that should be read when the execution resumes in the second script, after the user is redirected.

  • The second script is triggered after the external site redirects the user back to Cyclos. This script must return an HTML content which is shown to the user. After being redirected back to Cyclos, the previous web application state, such as breadcrumb, current page, etc., will be lost. Just the returned HTML content will be shown.

Bulk action

The script is executed for each user affected by the bulk action, and must return one of the following:

  • Null: The user was skipped;

  • Boolean: true represents the user was processed, false means the user was skipped;

  • org.cyclos.model.users.bulkactions.BulkActionUserStatus: The status of the user

  • String: If is the name of a BulkActionUserStatus enum item, is considered it. Otherwise, represents a message to be stored in the user log for this bulk action, and the user is assumed as processed;

  • Throwable: An error. Normally, errors are expressed by throwing exceptions, but it is also possible to return one;

  • org.cyclos.impl.users.BulkActionUserResult: A fully populated result.

Other script functions

Custom operations also support other script functions:

  • Code executed before the form is show, to fill the initial field values: This script will be executed before showing the form, so it can determine the default form fields dynamically. It should return a Map<String, Object>, containing, per custom field internal name, the initial value that should be presented for the user. Additionally, as part of the returned result object, you can specify the following properties:

    • autoRunAction: Either, the id or internal name of the action that should be executed automatically, instead of showing the form;

    • reRun: boolean indicating that the operation must be re-run when going back to it before displaying it. This will avoid showing outdated data when going back not from an operation action but using another mechanism (e.g the mobile app’s back button).

  • Code executed to determine whether the custom operation will be available: This script can decide to disable the custom operation from being shown as an option to be executed. For example, for record scope, some custom operation could only make sense if the record has a particular custom field value. If this script returns false, the operation will not be shown as an option. Any other value will enable the operation. This script will also run for custom operations used as actions (see below) of other custom operations. In this case, the formParameters bound variable will contain only the parameters that would be sent to the action, if it is executed, according to the mapped values in the action configuration. Also, for actions, an additional available variable for the script is containerCustomOperation, which is the main custom operation which is currently being executed, and that will contain the action;

  • Script code executed when the external site redirects the user back to Cyclos: This script is executed only if the operation result type is External redirect. It runs after the external service redirects the user back to Cyclos. The script will have access to the same storage object that was available to the main script block, so using that object, it is possible to pass data between both executions. The callback also runs with the same sessionData as the original script, they will have the same logged user, permissions, etc. Additionally, it is possible to read the original request parameters using the request variable.

Actions

Custom operations can have additional actions. Each action points to another custom operation with scope Internal. The original custom operation can be configured for which actions are available, which parameters are passed with that action and the visibility (when are shown to the user). Each parameter of the action operation may be mapped to a parameter of the original custom operation, or left for the original operation script to resolve the parameters that will be set to the action operation.

Actions can be configured on custom operations of result type 'Plain text', 'Rich text' or 'Result page'. The label defined for the custom operation pointed by an action will be used as the button label associated with that action, as the full name could be too large for buttons.

Also, actions can be configured to be displayed before executing the operation (for example, for result pages that can show a button to add a new row), after the execution or in both cases.

To control the actions, the object returned by the container custom operation can set a property named actions as part of the returned result. It should be a map keyed by the action operation internal name, and whose values contain the following properties:

  • parameters: Contains another map, keyed by parameter (form field of the action operation) internal name, with the value that should be used as that parameter. If there is a static mapping between an action parameter and an owner operation input field, and the script returns a parameter value, the script takes precedence;

  • enabled: Whether the action must be enabled or not. Default to true.

Actions shown before the execution of the original custom operation are customized by script executed before the form is shown, and those shown after are customized by the main script function.

Here is an example of a script for a custom operation of result type Rich text with two actions:

return [
    content: "This is the content displayed after the operation is executed",
    actions: [
        action1: [
            // action1 is the internal name of the custom operation
            // (with scope 'Internal') pointed by an action.
            parameters: [
                input1: "Value for input 1",
                input2: "1234" // Here input1 and input2 are internal names
                // of form fields in the action1 custom operation
            ]
            // action1 is enabled by default
        ],
        action2: [
            enabled: false // action2 is disabled for this execution
        ]
    ]
]

Here is an example of a script executed before show the form for a custom operation of result type Rich text with two actions (with visibility set to before or both):

return [
    // Here field1 is a form field in the custom operation
    field1: "Default value",
    actions: [
        action1: [
            // action1 is the internal name of the custom operation
            // (with scope 'Internal') pointed by an action.
            parameters: [
                input1: "Value for input 1",
                input2: "1234" // Here input1 and input2 are internal names
                // of form fields in the action1 custom operation
            ]
            // action1 is enabled by default
        ],
        action2: [
            enabled: false // action2 is disabled before the execution
        ]
    ]
]
Behavior after execution

Additionally, as part of the returned result object, you can specify what to do after a successful operation execution. Although you can specify this information for all result types, the Cyclos web application will process it only for Notification, URL (with 'newWindow' in true) and External redirect result types (except for autoRunAction). The following properties can be specified in the result:

  • backTo: Either the internal name, id or entity for the custom operation to which the user should be redirected after executing this operation. If such operation is in the current history (breadcrumb), the user will be redirected to it. Otherwise, the current page will not be changed;

  • backToRoot: A boolean value indicating if the application must go back to the page that originated the custom operation executions. If we already are in a 'root page' then the Cyclos web application will stay in the current page. For example, an operation with scope User containing an action (action 1) and this in turn containing another action (action 1.1) could generate the following history: View user profile → Run user custom operation → Run Custom operation action1 → Run Custom operation action1.1. In this case, the flag backToRoot set to true means go back to the 'View user profile' page;

  • reRun: A boolean value indicating if the page we went back to or the current one (if backTo was not specified or backToRoot is false) must be executed again before displaying it;

  • autoRunAction: Either the id or internal name of the action that should be executed automatically. If it is specified, the Cyclos web application doesn’t show the result and runs the action automatically, as it will if the user manually executes it by the corresponding action button.

Row actions on page results

Custom operations that return a page of results are very versatile. For example, they can be printed as PDF or exported to CSV / XLSX, or page results (if the script returns the total count).

Also, on the custom operation, it is possible to define an action to be executed when a row is clicked by the user. The possible actions are:

  • Navigate to an external URL: When clicking a row, the user is redirected to an external URL;

  • Navigate to a location in Cyclos: A list of common locations in Cyclos are presented;

  • Run an internal custom operation: Allows running a custom operation which has the scope = 'Internal'. This new operation will probably present some content to the user.

In all cases an action is set to a row, parameters can be passed to the next page. This is very important, as it will provide context on which data was selected. For an internal custom operation to receive a parameter, first on the result page custom operation the field 'URL parameters' must be set, having a comma-separated value of object properties to be passed to the internal custom operation. This will pass all such properties from the clicked row to the internal custom operation.

Then, the internal custom operation needs to have form fields defined with the matching internal name. Here is an example on this.

Receipt

Custom operations which return a content can also return a receipt to be printed in the Cyclos mobile application using a Bluetooth printer. The content of the receipt is simple, because the printing has limited style and layout possibilities.

The classic frontend for administrators has a button to preview the receipt on the browser. It is not a "pixel-perfect" version of the physical printing, but can aid developers of the script to have an acceptable version before testing it physically on paper in the mobile application.

To enable receipt printing, the receipt property should be present in the result. It should contain the following properties:

  • timestamp: When returned (normally with the value new Date()) the timestamp will be printed on the very beginning of the receipt;

  • header: Either a string or an object with the text displayed above the title, and below the timestamp. By default, the header is normal style and left aligned;

  • title: Either a string or an object with the title text displayed below the header and above the main items. By default, the title is bold, double height and center aligned;

  • items: A list of either strings or objects to be used as the main section. Each of these items either be a regular text or a label / value pair. See examples below;

  • footer: Either a string or an object with the text displayed at the end of the receipt. By default, the footer has lines before and after, is bold style and is center aligned;

  • labelSuffix: A string indicating a suffix for all labels in items. The default value is . It is possible to disable the suffix by returning a single space;

  • autoPrint: When set to true, the mobile app will start printing automatically.

These are the properties that can be used to configure each section or item:

  • label: Only for items. If set, the item enters into 'field' mode, with a left-aligned label and a right-aligned value, which is the text attribute. They are printed in the same line if both fit. Otherwise, the value is printed left-aligned in the next line;

  • labelStyle: Only for items with labels. If set, determine the font style of the label. Defaults to bold;

  • text: The section text;

  • style: Either normal, bold or underline;

  • align: Either left, center or right. Ignored for items with a label;

  • width: Font width. Either normal or double. Ignored for items with a label;

  • height: Font height. Either normal or double. Ignored for items with a label;

  • lineBefore: Boolean indicating whether a line should be printed before the text;

  • lineAfter: Boolean indicating whether a line should be printed after the text;

Here is an example of a script for a custom operation of result type 'Rich text' with a simple receipt:

return [
    content: "This is the content displayed after the operation is executed",
    receipt: [
        timestamp: new Date(),
        header: "This is the header content",
        title: "Transaction receipt",
        items: [
            "Thanks for this transaction!",
            "",
            // Blank line
            [
                label: "Field 1",
                text: "Value 1"
            ],
            [
                label: "Field 2",
                text: "Value 2"
            ]
        ],
        footer: "Hope to see you soon again!"
    ]
]

And here is another example, changing the defaults:

return [
    content: "This is the content displayed after the operation is executed",
    receipt: [
        timestamp: new Date(),
        header: [
            align: 'center',
            text: 'Centered header'
        ],
        title: [
            align: 'left',
            text: 'Left title'
        ],
        items: [
            [
                text: "On left"
            ],
            [
                text: "On right",
                align: "right"
            ]
        ],
        footer: [
            lineAfter: false,
            align: 'left',
            text: 'Left footer without bottom line'
        ],
    ]
]
Examples
Contact us page

This example allows creating a "contact us" page, which sends an e-mail to a specified address. To use it, you will need the following content in the script parameters box:

to=admin@project.org
from=noreply@project.org
subject=Contact form
message=The message was sent.\nThank you for your contact.

mailHeader=A user has sent a contact form with the following data:
mailFrom=From:
mailEmail=E-Mail:
mailSubject=Subject:
mailMessage=Message:

invalidEmail=Invalid e-mail address

Then, use the following script code:

import javax.mail.internet.InternetAddress

import org.cyclos.impl.utils.validation.validations.EmailValidation
import org.cyclos.model.ValidationException
import org.springframework.mail.javamail.MimeMessageHelper

def sender = mailHandler.mailSender
def message = sender.createMimeMessage()
def helper = new MimeMessageHelper(message)

if (!EmailValidation.isValid(formParameters.email)) {
    throw new ValidationException(scriptParameters.invalidEmail);
}

helper.to = new InternetAddress(scriptParameters.to)
helper.from = new InternetAddress(scriptParameters.from)
helper.subject = scriptParameters.subject
helper.text = """
${scriptParameters.mailHeader}
${scriptParameters.mailFrom} ${formParameters.from}
${scriptParameters.mailEmail} ${formParameters.email}
${scriptParameters.mailSubject} ${formParameters.subject}
${scriptParameters.mailMessage} ${formParameters.message}
"""
sender.send message

return scriptParameters.message

The custom operation needs form parameters with the following internal names: from, email, subject and message.

Returning a string (notification / rich / plain text)

Examples of a custom operation which returns a text (a notification in that case) can be found in the loan solution example.

Returning an external redirect

An example of an external redirect is the PayPal integration example.

Returning a file

This is an example where the user selects a document to download. It is assumed that the custom operation has a form field of type single selection with the internal name file. Then, each possible value should have the internal name corresponding to a PDF file in a given folder. Once the user chooses the file, it is downloaded.

import org.cyclos.model.ValidationException

// Assume there is a pdf file for each possible value of the field
String fileName = formParameters.file.internalName
String dir = scriptParameters.dir ?: "/usr/share/documents"
File file = new File(dir, "${fileName}.pdf")
if (!file.exists()) {
    throw new ValidationException("File not found")
}
return [
    content: file,
    contentType: "application/pdf",
    name: file.name,
    length: file.length(),
    lastModified: file.lastModified()
]
View users I’ve traded with

In this example, a user can see the other users he has traded with (either performed or received payments). The custom operation needs to have 'User' scope and 'Result page' as result type. Also, it needs to have the URL action as 'Cyclos location', and the location needs to be user_profile. Finally, set as 'URL parameters' the value id. For more details, see the next section.

import org.cyclos.impl.banking.AccountServiceLocal
import org.cyclos.model.ValidationException
import org.springframework.jdbc.core.ColumnMapRowMapper
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate

AccountServiceLocal accountService = binding.accountService

List<Long> accountIds = accountService.listVisible(user).collect {acc -> acc.id}
if (accountIds.empty) {
    throw new ValidationException("No accounts")
}

NamedParameterJdbcTemplate jdbc = binding.namedParameterJdbcTemplate

// First count the number of users / currencies that traded with the current user
Integer totalCount = null
if (!skipTotalCount) {
    totalCount = jdbc.queryForObject("""
        select count(*)
        from (
            select distinct user_id, currency_id
            from (
                select user_id, currency_id
                from (
                    select u.id as user_id, at.currency_id, max(t.date) as last_date, max(t.amount) as max_amount, count(*) as count
                    from transfers t inner join accounts a on t.to_id = a.id
                    inner join users u on a.user_id = u.id
                    inner join account_types at on a.account_type_id = at.id
                    where t.from_id in (:accountIds)
                    group by u.id, at.currency_id
                    union
                    select u.id as user_id, at.currency_id, max(t.date) as last_date, max(t.amount) as max_amount, count(*) as count
                    from transfers t inner join accounts a on t.from_id = a.id
                    inner join users u on a.user_id = u.id
                    inner join account_types at on a.account_type_id = at.id
                    where t.to_id in (:accountIds)
                    group by u.id, at.currency_id
                ) t1
                group by user_id, currency_id
            ) t2
        ) t3
    """, [accountIds: accountIds], Integer);
}

// Then get the data
int pageSize = binding.pageSize
int currentPage = binding.currentPage
def rows = jdbc.query("""
    select u.id, u.display_for_managers, t.currency_id as "currencyId", t.last_date as "lastDate", t.max_amount as "maxAmount", t.count
    from (
        select user_id, currency_id, max(last_date) as last_date, max(max_amount) as max_amount, sum(count) as count
        from (
            select u.id as user_id, at.currency_id, max(t.date) as last_date, max(t.amount) as max_amount, count(*) as count
            from transfers t inner join accounts a on t.to_id = a.id
            inner join users u on a.user_id = u.id
            inner join account_types at on a.account_type_id = at.id
            where t.from_id in (:accountIds)
            group by u.id, at.currency_id
            union
            select u.id as user_id, at.currency_id, max(t.date) as last_date, max(t.amount) as max_amount, count(*) as count
            from transfers t inner join accounts a on t.from_id = a.id
            inner join users u on a.user_id = u.id
            inner join account_types at on a.account_type_id = at.id
            where t.to_id in (:accountIds)
            group by u.id, at.currency_id
        ) t1
        group by user_id, currency_id
    ) t inner join users u on t.user_id = u.id
    order by t.count desc, u.display_for_managers
    limit :limit
    offset :offset
""", [accountIds: accountIds, limit: pageSize + 1, offset: pageSize * currentPage], new ColumnMapRowMapper());

// Build the result
return [
    columns: [
        [header: "User", property: "display_for_managers", width: "40%"],
        [header: "Last date", property: "lastDate", align: "center", type: "date", width: "20%"],
        [header: "Max amount", property: "maxAmount", currencyProperty: "currencyId", align: "right", width: "20%"],
        [header: "Transactions", property: "count", align: "right", type: "number", width: "20%"],
    ],
    rows: rows,
    totalCount: totalCount
]
Search records with an empty field value

In this example, an administrator can search for user records which don’t have a specific custom field value. The record search page allows filtering by values, but not by records without value for a particular field.

The custom operation needs to have system scope and result type result page. You can also set the "Action when clicking a row" to "Navigate to a Cyclos location", set "Location" to record and set "Parameters to be passed (comma-separated names)" to id.

import org.cyclos.entities.users.QRecordCustomFieldValue
import org.cyclos.entities.users.QUserRecord
import org.cyclos.entities.users.RecordCustomField
import org.cyclos.entities.users.UserRecord
import org.cyclos.entities.users.UserRecordType
import org.cyclos.impl.utils.persistence.DBQuery
import org.cyclos.model.general.GeneralKeys
import org.cyclos.model.system.fields.CustomFieldType
import org.cyclos.model.users.UsersKeys
import org.cyclos.utils.Page
import org.cyclos.utils.PageImpl

def r = QUserRecord.userRecord
def v = QRecordCustomFieldValue.recordCustomFieldValue

// Read the record type and custom field (adjust the internal names)
def recordType = entityManagerHandler.find(UserRecordType, scriptParameters.recordType)
def field = entityManagerHandler.find(RecordCustomField, scriptParameters.field, recordType)

DBQuery query = entityManagerHandler.from(r)
        .leftJoin(v).on(v.owner().eq(r), v.field().eq(field))
        .where(r.type().eq(recordType))

// According to the custom field type, the condition for empty changes
switch (field.type) {
    case CustomFieldType.STRING:
    case CustomFieldType.URL:
    case CustomFieldType.DYNAMIC_SELECTION:
        query.where(v.stringValue.isNull().or(v.stringValue.isEmpty()))
        break
    case CustomFieldType.TEXT:
        query.where(v.textValue.isNull().or(v.textValue.isEmpty()))
        break
    case CustomFieldType.RICH_TEXT:
        query.where(v.richTextValue.isNull().or(v.richTextValue.isEmpty()))
        break
    case CustomFieldType.DATE:
        query.where(v.dateValue.isNull())
        break
    case CustomFieldType.INTEGER:
        query.where(v.integerValue.isNull())
        break
    case CustomFieldType.DECIMAL:
        query.where(v.decimalValue.isNull())
        break
    case CustomFieldType.LINKED_ENTITY:
        query.where(v.linkedEntityId.isNull())
        break
    case CustomFieldType.SINGLE_SELECTION:
        query.where(v.enumeratedValue.isNull())
        break
    case CustomFieldType.MULTI_SELECTION:
        query.where(v.enumeratedValues.isEmpty())
        break

    default:
        throw new IllegalStateException("Unhandled custom field type: ${field.type}")
}

query.orderBy(r.creationDate.desc())

// Execute the query
def page = query.page(currentPage, pageSize, skipTotalCount, r) as Page<UserRecord>
def projection = {
    [
        id: it.id,
        date: it.creationDate,
        user: it.user.displayForManagers,
        // This will use the record type's 'Display records as',
        // which can be set to something like: "{fieldA} of {fieldB}"
        // and defaults to: "{type} ({id})"
        record: formatter.format(it)
    ]
}

// Return the records without value
return [
    columns: [
        [header: translationHandler.message(UsersKeys.Records.CREATION_DATE), property: 'date', type: 'date'],
        [header: translationHandler.message(UsersKeys.Records.USER), property: 'user'],
        [header: translationHandler.message(GeneralKeys.InitialData.RECORD_TYPE), property: 'record']
    ],
    rows: PageImpl.transformed(page, projection),
    totalCount: page.totalCount,
    hasNextPage: page.hasNextPage
]

In the custom operation, besides setting the script, also set 2 parameters in the "Script parameters" field:

  • recordType: internal name of the record type to search;

  • field: internal name of the custom field to search.

This example presents users a link and QR-code which can be shared for other users to pay him / her (easy invoice). The QR-code can be scanned by the Cyclos mobile application from the payer. To use it, you will need the following content in the script parameters box:

## The message shown above
message=You can copy and share the following easy invoice link or QR-code, \
which can be scanned by the Cyclos mobile application:

## Currency to be appended to the URL.
## Not needed if users have a single currency.
# currency=unit

## Payment type to be appended to the URL.
## Not needed if users have a single payment type.
paymentType=user.tradeTransfer

Then, use the following script code:

import org.apache.commons.text.StringEscapeUtils
import org.cyclos.utils.StringHelper

def rootUrl = sessionData.configuration.fullUrl

// Get the amount
def amount = formParameters.amount.toPlainString()

// Get the description
def description = formParameters.description

// Get the to user his username
def to = user.username

def parameters = "&amount=${amount}"
if (StringHelper.isNotBlank(description)) {
    description = StringHelper.encodeURIComponent(description)
    description = StringHelper.replace(description, "+", "%20")
    parameters += "&description=${description}"
}
if (StringHelper.isNotBlank(scriptParameters.currency)) {
    parameters += "&currency=${scriptParameters.currency}"
}
if (StringHelper.isNotBlank(scriptParameters.paymentType)) {
    parameters += "&type=${scriptParameters.paymentType}"
}

def url = "${rootUrl}/pay/?to=${to}${parameters}"
def qrCode = "${rootUrl}/api/tickets/easy-invoice-qr-code/*:${to}?size=medium${parameters}"

// Return the result
return """
<p>${scriptParameters.message}</p>
<p>
    <br>
    <a href="${url}" target="_blank">${StringEscapeUtils.escapeHtml4(url)}</a>
</p>
<p style="text-align:center">
    <br>
    <img src="${qrCode}">
</p>
"""

Then create the custom operation:

  • Name: Easy invoice;

  • Script: Select the Get easy invoice link / QR-Code script;

  • Scope: User;

  • Result type: Rich text.

    Finally, after saving, add the following form parameters:
  • Amount:

    • Internal name: amount;

    • Data type: Decimal;

    • Decimal digits: 2 (adjust according to the currency);

    • Required: Yes.

  • Description:

    • Internal name: description;

    • Data type: Multi-line text;

    • Required: No.

Loan request (content page with action)

This example shows some input fields for users to request a loan. Then shows the loan details and an action for the user to send the loan application. When clicked, an e-mail is sent with the request data, so the administration can actually handle that loan.

As both scripts calculate the loan, another script of type library is used. It contains a parameter for the interest rate. So, first is the code for the "Loan application" library script:

import java.math.RoundingMode

import org.cyclos.entities.users.User

import groovy.xml.MarkupBuilder

/**
 * Calculates the installment amount by composite monthly interests
 */
def installmentAmount(double rate, double totalAmount, int installments) {
    rate /= 100.0
    double cf = rate / (1 - (1 / Math.pow(1 + rate, installments)))
    return new BigDecimal(totalAmount * cf).setScale(2, RoundingMode.HALF_UP)
}

/**
 * Returns an HTML with the loan request details
 */
String loanRequestHTML(double rate, double reqAmount, int installments, User user) {
    def instAmount = installmentAmount(rate, reqAmount, installments)
    def totalAmount = instAmount * installments
    def out = new StringWriter()
    MarkupBuilder html = new MarkupBuilder(out)
    html.div {
        table {
            if (user != null) {
                tr {
                    td width:"200px", { b "Requested by user" }
                    td "${user.name} (${user.username})"
                }
            }
            tr {
                td width:"200px", { b "Monthly interest rates" }
                td "${formatter.format(rate as BigDecimal)}% per month"
            }
            tr {
                td { b "Requested amount" }
                td formatter.format(reqAmount, 2)
            }
            tr {
                td { b "Total amount to be repaid" }
                td formatter.format(totalAmount as BigDecimal, 2)
            }
            tr {
                td colspan: 2, { b "Installments"
                } }
            tr {
                td style:"text-align:center", { b "Due date" }
                td style:"text-align:right", { b "Due amount" }
            }
            def cal = Calendar.getInstance()
            for (int i = 0; i < installments; i++) {
                cal.add(Calendar.MONTH, 1)
                tr {
                    td style:"text-align:center", { b formatter.formatAsDate(cal.time) }
                    td style:"text-align:right", formatter.format(instAmount, 2)
                }
            }
        }
    }
    return out.toString()
}

This script needs a parameter which is the interest rate. So, paste this in the script parameters field:

monthlyInterests=0.75
email=admin@admin-email.com

Here is the code for the custom operation that requests the loan. Don’t forget to include the loan application library in the script.

def rate = scriptParameters.monthlyInterests as double
def reqAmount = formParameters.amount as BigDecimal
def instCount = formParameters.installments as int

return [
    title: "Loan request details",
    content: loanRequestHTML(rate, reqAmount, instCount, null),
    actions: [
        submitLoanApplication: [
            parameters: [
                user: user.id
            ]
        ]
    ]
]

And here is the code for the custom operation that submits the loan request. The script should also include the loan application library.

import javax.mail.internet.InternetAddress

import org.springframework.mail.javamail.MimeMessageHelper

def rate = scriptParameters.monthlyInterests as double
def reqAmount = formParameters.amount as BigDecimal
def instCount = formParameters.installments
def user = formParameters.user
def body = loanRequestHTML(rate, reqAmount, instCount, user)

def sender = mailHandler.mailSender
def message = sender.createMimeMessage()
def helper = new MimeMessageHelper(message, true, "UTF-8")

helper.to = new InternetAddress(scriptParameters.email)
helper.from = new InternetAddress(user.email, user.name)
helper.subject = "Loan request"
helper.setText(body, true)
sender.send message

return "The loan request was sent to the administration"

Before creating the custom operation for the loan application itself, create the one for the action, with the following fields:

  • Name: Send loan application;

  • Internal name: submitLoanApplication;

  • Label: Send;

  • Script: Select the one with the code to send the application;

  • Main menu: Banking;

  • Scope: Internal;

  • Result type: Notification.

    Then, after saving, add the following form parameters:
  • Total amount:

    • Internal name: amount;

    • Data type: Decimal;

    • Decimal digits: 2;

    • Required: Yes.

  • Number of installments:

    • Internal name: installments;

    • Data type: Integer;

    • Required: Yes.

  • User:

    • Internal name: user;

    • Data type: Linked entity;

    • Linked entity type: User;

    • Required: Yes.

Then create the custom operation with the loan application form:

  • Name: Loan application;

  • Internal name: loanApplication;

  • Script: Select the loan application request script;

  • Scope: user;

  • Result type: Rich text.

Then, after saving, add the following form parameters:

  • Total amount:

    • Internal name: amount;

    • Data type: Decimal;

    • Decimal digits: 2;

    • Required: Yes.

  • Number of installments:

    • Internal name: installments;

    • Data type: Integer;

    • Required: Yes.

And add another action in the actions tab:

  • Total amount: Map to this operation’s Total amount;

  • Number of installments: Map to this operation’s Number of installments;

  • User: Set it for the script to define the value.

After granting permission to the Loan application custom operation, it should appear in the menu.

Searching external records

The following is an example script for a custom operation which lists fictional external records. It needs to have as URL action the custom operation presented ahead to show an external record details, and pass the URL parameter recordId:

return [
    columns: [
        [header:"Name", property:"name"]
    ],
    rows: [
        [name: "Record 1", recordId: 1],
        [name: "Record 2", recordId: 2],
        [name: "Record 3", recordId: 3],
        [name: "Record 4", recordId: 4],
        [name: "Record 5", recordId: 5],
        [name: "Invalid Record", recordId: 99999],
    ]
]

Then another custom operation, which should be defined as internal, and have a form field which internal name recordId:

import org.cyclos.model.EntityNotFoundException

// Validate the id
def recordId = formParameters.recordId
def validIds = 1..50
if (!(recordId in validIds)) {
    throw new EntityNotFoundException([
        entityType: "External record",
        key: recordId as String])
}

return [
    title: "Details for record ${recordId}",
    content: "This is the description for record ${recordId}"
]
Bulk action to perform a payment from system

This is an example of a bulk action custom operation that performs a payment from a system account to each processed user. The script checks if the user has the account that receives the payment. If not, it is marked as skipped for that bulk action. If the user has the account, the payment is performed.

The script needs as parameters, the internal name of the system account and payment type, like this (make sure to check that the internal names are correct):

systemAccount=debit
paymentType=toUser

Then the custom operation script block should be as follows:

import org.cyclos.entities.banking.TransferType
import org.cyclos.model.EntityNotFoundException
import org.cyclos.model.banking.accounts.SystemAccountOwner
import org.cyclos.model.banking.transactions.PerformPaymentDTO
import org.cyclos.model.banking.transfertypes.TransferTypeVO
import org.cyclos.model.users.bulkactions.BulkActionUserStatus

def tt = entityManagerHandler.find(TransferType,
        "${scriptParameters.systemAccount}.${scriptParameters.paymentType}")

// Check if the user has the destination account type
try {
    accountService.load(user, tt.to)
} catch (EntityNotFoundException e) {
    return BulkActionUserStatus.SKIPPED
}

// Perform the payment
def dto = new PerformPaymentDTO()
dto.owner = SystemAccountOwner.instance()
dto.subject = user
dto.type = new TransferTypeVO(tt.id)
dto.amount = formParameters.amount
paymentService.perform(dto)
return BulkActionUserStatus.SUCCESS
Bulk action to remove canceled tokens

In this example a bulk action is used to remove canceled tokens, optionally filtering by type. If a type is selected, only the canceled tokens of that type will be removed, otherwise all canceled tokens will be removed.

First, create the script to load token principal types (of type load custom field values):

import org.cyclos.model.system.fields.DynamicFieldValueVO

return principalTypeService.listUserTokenPermissions(null).collect {
    new DynamicFieldValueVO(it.type.internalName, it.type.name)
}

And the script for the custom operation:

import org.cyclos.model.access.principaltypes.TokenPrincipalTypeVO
import org.cyclos.model.access.tokens.TokenQuery
import org.cyclos.model.access.tokens.TokenStatus
import org.cyclos.model.users.users.UserVO
import org.cyclos.model.utils.ModelHelper
import org.cyclos.utils.CollectionHelper

def query = new TokenQuery()
query.setUnlimited()
query.setUser(conversionHandler.convert(UserVO.class, user))
query.setStatuses(CollectionHelper.asSet(TokenStatus.CANCELED))
if (formParameters.tokenType) {
    query.setType(ModelHelper.voFromString(TokenPrincipalTypeVO.class, formParameters.tokenType.value))
}

tokenService.search(query).getPageItems().each{ tokenService.remove(it.getId()) }

return "Tokens removed successfully"

Then create the custom operation:

  • Name: Remove canceled tokens (Can be changed);

  • Scope: Bulk action;

  • Script: Remove canceled tokens (the script created above);

  • Show form: Always.

Finally, after saving, add the following form parameter:

  • Display name: Token type;

  • Internal name: tokenType;

  • Data type: Dynamic selection;

  • Load values script: Load token types (The first script created);

  • Field type: Dropdown;

  • Required: No (Set it in "Yes" if you don’t want to let the user who runs the bulk action remove tokens of all types at once).

Mobile app rating

This example allows rating the mobile app in Google Play and App Store

First create the script which handles the link redirection, this will open up in the store where the user can give an app rating

def userAgent = userAgentHandler.parse(sessionData.requestData.requestInfo)
def platform = userAgent != null ? userAgent.operatingSystem.toLowerCase() : null

if (platform == 'android') {
    // Replace <package-name> with your Android application package
    return 'market://details?id=<package-name>'
} else if (platform == 'ios') {
    // Replace <app-id> with your id from the App Store
    return 'itms-apps://itunes.apple.com/app/id<app-id>'
}

// We should never reach here when connecting from a mobile device
return ""

Then create the custom operation, in case you prefer to show the action in the Mobile app homepage use scope 'User' or in case you want to rate after a payment use scope 'Transfer':

  • Name: Rate app;

  • Scope: User / Transfer;

  • Enabled for channels: Mobile app;

  • Script: Rate app (the script created above);

  • Result type: URL.

Sending mobile app notifications

The following example shows how to send a mobile app notification directly to a user. Such notifications will be shown to the user even if the mobile application is not running or is running in background. The underlying implementation uses Firebase Cloud Messaging (FCM) to send the notifications. How to configure FCM is included in the documentation of the mobile application, if you do not have access to it please contact STRO at info@cyclos.org.

Before continue, please check the support Cyclos already has for mobile app mailings and also for sending all internal notifications as mobile app notifications. See this to know how to customize the notifications.

You should consider this example only if the above options do not fulfil your particular requirements.

The script below assumes you created a custom operation with scope User associated with it.

import org.cyclos.impl.system.ScriptHelper
import org.cyclos.impl.utils.appnotifications.AppNotificationHandler
import org.cyclos.impl.utils.appnotifications.AppNotificationMessage

ScriptHelper scriptHelper = binding.scriptHelper
// Get a reference to the app notification handler
AppNotificationHandler appNotificationHandler = scriptHelper.bean(AppNotificationHandler)

if (appNotificationHandler.canReceiveNotifications(user)) {
    def builder = AppNotificationMessage.builder()
    builder.user = user
    builder.title = "Sample title"
    builder.body = "Sample body"
    // (Optional) You can set the notification icon color (RGB format) to be shown in the notification drawer (only for Android). Default: null
    // builder.androidIconColor = '#f79734'

    // (Optional) You can set the public URL of a custom system/user image. Default: null
    //builder.imageUrl =

    // (Optional) Whether to show a badge for the iOS notification or not. Default: true
    //builer.iosBadge = true

    //(Optional) Instruct to go to the profile page when the user tap in the notification. Default: empty
    builder.variables = [
        customUrl: 'cyclos://profile',
        userId: scriptHelper.maskId(user.getId()).toString()
    ]
    // Send the notification
    return appNotificationHandler.notify(builder.build()) ?
            "Notification sent to ${user.username}" :
            "The notification could not be sent to ${user.username}"
} else {
    return "User ${user.username} can not receive notifications!"
}

4.4.12. Custom wizards

These scripts are invoked when a user runs a custom wizard. A custom wizard can be of the following types:

  • Registration: Replaces the registration form. Gives the opportunity to present custom fields defined in the wizard itself, allowing more data to be collected and processed by script. Besides creating the script and the wizard, in the Configuration menu, the wizards should be set for registration on large screens (desktops), medium screens (tablets) and small screens (phones). When all 3 are set, the regular registration form / API is disabled;

  • User: The wizard is executed by users via a menu item. Can also be set for administrators and brokers to run over other users via the profile;

  • System: The wizard is executed by administrators via a menu item;

  • Guest: The wizard is executed by guests. Can be shown in a menu via the menu entries in content management.

A wizard consists of several steps, which are manually ordered. When the wizard starts, by default the first step is shown. On each transition, by default the next step is executed, until the last step. After finishing the wizard, a result is shown.

Script can control which is the first step, and which are the possible transitions between steps. The transitions are determined before the step is shown, because each possible transition is displayed as a different button to users. When the script doesn’t return any transitions, the default is to use a single transition to the next step in the defined order.

Additional bound variables

In all cases, the script will have the following bound variables (besides the default bindings):

  • wizard: The org.cyclos.entities.system.CustomWizard being executed;

  • execution: The org.cyclos.entities.system.CustomWizardExecution which contains all data for a given wizard execution, and is shared between steps;

  • step: The current org.cyclos.entities.system.CustomWizardStep. On transition, is the candidate next step;

  • previousStep: The previous org.cyclos.entities.system.CustomWizardStep. Only available in the transition function;

  • transition: The transition id the user has selected. Only present on transitions;

  • user: The org.cyclos.entities.users.User. The meaning depends on the wizard type. For registration wizards, it is only present in the finish function, and it is the newly registered user. For user wizards it is the user over which it is being executed;

  • steps: The list of all steps, together with the possible transitions executed so far. Is a list of org.cyclos.impl.system.CustomWizardStepWithTransitions;

  • storage: A org.cyclos.impl.system.CustomWizardExecutionStorage which is shared between all steps. Scripts can use it to pass data between steps, and to the final execution script function;

  • registration: The org.cyclos.model.users.users.PublicRegistrationDTO which is filled-in on each step. Only present for registration wizards. To persist any modification, assign it back to the storage, using storage.registration = registration;

  • customValues: A Map<String, Object> keyed by custom field internal name, whose values are the values filled-in during the execution. Do not confuse it with custom profile fields, which are stored in the registration object. To persist any modification, assign it back to the storage, using storage.customValues = customValues;

  • returnUrl: The URL to pass to the external system on external redirects. Is the URL to which the external system should redirect the user when returning to Cyclos;

  • request: The org.cyclos.model.utils.RequestInfo. Only present on the script that runs the callback after an external redirect. Contains the information about the current request, so the script function which handles the callback can identify the context to complete the process.

Finish

This is the only required code block. The script is executed after finishing the last step. For registration wizards, the script is executed after the user has been registered, so additional actions can be performed on that user, and the result is ignored. For other wizards, this is the action executed on finish.

Script result
  • String: The result content, which is handled either as plain text or HTML, depending on the wizard configuration;

  • Object or Map: With the following properties

    • title: The page title displayed on result;

    • result: The result content.

New execution

This code is executed whenever a new execution starts. It is used to determine the initial step.

Script result
  • Null: The first step in order ise used as the initial step. It will have a single transition, to the subsequent step in order;

  • String: The internal name of a step. It will have a single transition, to the next step in order;

  • org.cyclos.entities.system.CustomWizardStep: A reference to the initial step. It will have a single transition, to the next step in order;

  • Object or Map: With the following properties:

    • step: Either a string with the internal name or the initial step itself;

    • transitions: A single value or collection of the possible transitions. Each element should be an object or Map with the following properties:

      • id: The transition id. This string is passed by clients when transitioning between steps, and is passed to the function that handles transitions;

      • label: The label displayed in the transition button.

Transition between steps

This code is executed whenever an execution is transitioned between steps. The script determines which is the next step and its possible new transitions for subsequent steps. The result is the same as the script block above. The difference is that when nothing is returned, the candidate next step is actually used.

WARNING: A change was introduced in Cyclos 4.16 for this script. On previous versions, the next step must have been included in the transition, whereas starting in 4.16, only a transition id and a label are used. This allows the script to determine the next step dynamically, for example, based on a custom field presented in the previous step. The expected result also changed. Before, the step was predefined, only the possible transitions were returned. Now, the script must return which is the next step and, optionally, its transitions.

Redirect to external site

This code is executed when the user confirms a step which is configured as external redirect. It is used to interact with an external system during the wizard. For example, a top-up could be required during the registration process. Please, note the returnUrl bound variable, which is the URL to pass to the external system, indicating where the external system should redirect the user back to Cyclos.

Script result
  • String: The URL of the external service to which the user will be redirected.

External site callback

This code is executed after the external redirect completes.

Script result
  • Null or True: The execution will automatically transition to the next step in the defined order;

  • False: Is interpreted as canceling the external redirect action. The execution will stay in the current step.

  • String: Is handled as the identifier of the transition for the next step.

Tips
  • Use the storage variable to store and retrieve custom data on any step of the current execution. The storage also provides access to specialized data within the execution. See the class JavaDoc for more details.

  • The script can send notifications, which are displayed in the current step. For that, call either storage.info(message), storage.warn(message) or storage.error(message). Once a transition happens, any pending notifications are cleared.

  • If the wizard has a step that redirects to an external site, that step cannot be the last one. That is because after the redirect back to Cyclos, the wizard will transition automatically to the next step. The wizard execution page will have to query the current status. If the execution had ended, all the context would have been lost for that execution.

Examples
Registration with a required top-up

This example requires a top-up via PayPal for the public user registration. It uses the same PayPal library from PayPal Integration. Make sure you have the library code updated.

The example uses 3 script blocks for the wizard script, plus script parameters.

Script parameters:

# Settings for the access token record type
auth.recordType = paypalAuth
auth.clientId = clientId
auth.clientSecret = clientSecret
auth.token = token
auth.tokenExpiration = tokenExpiration

# Settings for PayPal
mode = sandbox
currency = EUR
paymentDescription = Initial top-up

# Settings for the Cyclos payment
amountMultiplier = 1
accountType = debitUnits
paymentType = paypalCredits

# Messages
error.invalidRequest = Invalid request
error.transactionNotFound = Transaction not found
error.transactionAlreadyApproved = The transaction was already approved
error.payment = There was an error while processing the top-up. Please, try again.
error.notApproved = The top-up was not approved
message.canceled = The top-up was canceled
message.done = The top-up was approved

Script code executed when the wizard finishes:

import org.cyclos.entities.users.User
import org.cyclos.impl.system.CustomWizardExecutionStorage
import org.cyclos.impl.system.ScriptHelper
import org.cyclos.model.ValidationException

import groovy.transform.TypeChecked

@TypeChecked
def performPayments() {
    def variables = binding.variables as Map<String, Object>
    def scriptParameters = variables.scriptParameters as Map<String, String>
    def service = new PayPalService(variables)
    def storage = variables.storage as CustomWizardExecutionStorage
    def scriptHelper = variables.scriptHelper as ScriptHelper
    def user = variables.user as User
    def orderId = storage.getString('payPalOrderId')

    // If no order id, return an error
    if (orderId == null) {
        throw new ValidationException('No PayPal payment data')
    }

    def order = service.getOrderFromPayPal(orderId)
    if(order.status == "APPROVED") {
        // Add a commit listener to perform the payments,
        // it will be executed after a successful registration
        scriptHelper.addOnCommitTransactional({
            // Execute the PayPal payment
            def capturedOrder = service.captureOrder(orderId)
            try {
                // Try to perform the payment in Cyclos,
                // if fails, refund the payment in PayPal
                service.perform(capturedOrder, user)
            } catch (Exception ex) {
                service.refundCapturedOrder(capturedOrder, null, user)
            }
        })
    } else {
        throw new ValidationException(scriptParameters.'error.notApproved'
        ?: "The payment was not approved")
    }
}

performPayments()

Script code executed before the user is redirected to an external site:

import org.cyclos.impl.system.CustomWizardExecutionStorage

import groovy.transform.TypeChecked

@TypeChecked
def createOrder(){
    def variables = binding.variables
    def service = new PayPalService(variables)
    def storage = variables.storage as CustomWizardExecutionStorage

    def customValues = variables.customValues as Map<String, Object>
    def amount = customValues.amount as Number
    def returnUrl = variables.returnUrl as String

    def order = service.createOrder(amount, returnUrl)
    def link = (order.links as Map<String, Object>[])
            .find {it.rel == "approve"}
    if (link) {
        // Store the returned order id
        storage.setString('payPalOrderId', order.id as String)
        return link.href
    } else {
        throw new IllegalStateException("No approval url returned from PayPal")
    }
}

createOrder()

Script code executed when the external site redirects the user back to Cyclos:

import org.cyclos.impl.system.CustomWizardExecutionStorage
import org.cyclos.model.ValidationException
import org.cyclos.model.utils.RequestInfo

import groovy.transform.TypeChecked

@TypeChecked
def payPalCallback() {
    def variables = binding.variables as Map<String, Object>
    def scriptParameters = variables.scriptParameters as Map<String, String>
    def storage = variables.storage as CustomWizardExecutionStorage
    def request = variables.request as RequestInfo

    if (request.getParameter('cancel')) {
        // The operation has been canceled. Don't transition
        storage.warn(scriptParameters.'message.canceled'
                ?: 'The top-up was canceled')
        return false
    }
    // If no order id, return an error
    if (storage.getString('payPalOrderId') == null) {
        throw new ValidationException('Invalid request')
    }
}

payPalCallback()

Then, in your registration wizard, create a custom field with internal name amount, of type 'Decimal', and required. Assign that field to the wizard step that performs an external redirect (and hence, cannot be the last one).

Deciding the next step dynamically

This example is for the "Script code executed on transitions between steps" to dynamically choose the next step.

// Note that this example works for Cyclos 4.16 onwards
if (previousStep.internalName == 'typeSelection') {
    // Example of transition based on the previous step
    def type = customValues.type.internalName
    // A custom field shown in the previous step is used to decide the next step
    return [
        step: type == 'business' ? 'businessFields' : 'consumerFields',
        transitions: 'details'
    ]
} else if (step.internalName == 'details') {
    // Example of a transition based on the candidate next step
    def formType = customValues.formType.internalName
    // Skip the details step if on simple form and continue to confirmation
    return formType == 'simple' ? 'confirmation' : step.internalName
}

4.4.13. Custom web services

These scripts are invoked when a request is received in some path under <cyclos-root-url>[/network]/run/*. To actually run them, it is needed to create a custom web service definition in the System - Tools - Custom web services menu.

The custom web services have the following important properties:

  • The accepted HTTP methods: GET, POST or both;

  • Whether the script will be executed as a guest (optionally using a fixed HTTP username / password, with basic authorization) or as an authenticated user, like with other web services, using the same headers described in authentication in web services;

  • When marked to be executed as user, it is required to grant the user permission to run it, in the product;

  • An IP address whitelist, to control which hosts can call the custom web service;

  • The URL mappings, which is a list of paths (one per line) to be matched after the <cyclos-root-url>[/network]/run root path. It is possible to specify the following types of paths:

    • Simple paths. For example, users, matches <cyclos-root-url>[/network]/run/users;

    • Nested paths. For example, users/list, matches <cyclos-root-url>[/network]/run/users/list;

    • Wildcards. For example, users/*, matches <cyclos-root-url>[/network]/run/users/a, but not <cyclos-root-url>[/network]/run/users/a/b;

    • Nested wildcards. For example, users/**, matches <cyclos-root-url>[/network]/run/users/a/b/c;

    • Path variables. For example, users/{groupId}/{userId}, matches <cyclos-root-url>[/network]/run/users/123/78, and the pathVariables variable will be available to the script with the value {groupId:123,userId:78}.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

  • customWebService: The org.cyclos.entities.system.CustomWebService being executed;

  • request: The org.cyclos.model.utils.RequestInfo which allows reading the original request parameters, headers and so on;

  • path: A string containing the path part after the <cyclos-root-url>[/network]/run/ prefix. Is neither initiated or terminated with /;

  • pathVariables: A Map<String,String> containing the matched path variable values.

Script result
  • A org.cyclos.model.utils.ResponseInfo, containing full information to build the HTTP response;

  • Null: The response will have status code 200 and no body;

  • String: The response will have status code 200, Content-type: text/plain`, and the returned string as body;

  • Object or Collection: The response will have status code 200, Content-type: application/json, and the body will contain a JSON representation of the returned object.

Handling exceptions

If the script captures an exception and wants to customize the response, instead of silencing the exception in a catch clause and returning a org.cyclos.model.utils.ResponseInfo, which will cause the current transaction to commit, possibly leaving the database in an inconsistent state, the script should throw a org.cyclos.model.utils.ResponseException, which contains a ResponseInfo internally. This way the main transaction is rolled back.

Other exceptions than `ResponseException`s are returned as HTTP status codes other than 200, and the details are returned as JSON.

Permissions

Sometimes it is useful to extend the Cyclos API to clients, like doing specific payments, or running a series of operations in a single request. However, it is important to use the same permissions as the user would normally have, to prevent security breaches. To do so, 3 steps are needed:

  • Make sure the script uses the security layer: Whenever using a service, use the security layer instead of the direct service implementation. For example, use the userServiceSecurity variable instead of userService, which would completely bypass security checks;

  • Ensure the custom web service has user authentication: On the custom web service details page, ensure it runs as user, not as guest. Also grant permission for the custom web service in the products page;

  • On the script, make sure it runs with the user permissions: On the details page of the script used by the custom web service, make sure the checkbox called Run with all permissions is unchecked. This guarantees the script will run with the exact permissions as the user.

Examples
Perform a payment

This example allows a caller to quickly perform a payment between 2 users. It is assumed that the URL mapping is something like payment/{from}/{to}/{amount} and there is a single possible payment type between the 2 users.

import org.cyclos.model.banking.transactions.PerformPaymentDTO
import org.cyclos.model.users.users.UserLocatorVO

def pmt = new PerformPaymentDTO()
pmt.owner = new UserLocatorVO(principal: pathVariables.from)
pmt.subject = new UserLocatorVO(principal: pathVariables.to)
pmt.amount = pathVariables.getDecimal('amount')

// Perform the payment and return the complete PaymentVO
return paymentService.perform(pmt)
Single-sign-on (login users without their passwords)

With this example, it is possible to login a user (create a session) without their password. This is useful when Cyclos works as a single-sign-on, with the user authenticated by some other system.

Just be extra careful with the external security which will be employed, such as creating an IP address whitelist, a guest user / password, etc. on the custom web service, otherwise, anyone could impersonate any user.

The script receives 2 query parameters: user, which is the login name (or some other identification, such as e-mail) of the user to be logged in, and remoteAddress, which is the remote IP address of the client accessing the third party software.

The script code is the following:

import org.cyclos.impl.access.SessionDataFactory
import org.cyclos.impl.access.SessionHandler.CreateSessionParameters
import org.cyclos.model.access.RequestData
import org.cyclos.model.access.channels.BuiltInChannel
import org.cyclos.model.users.users.UserLocatorVO
import org.cyclos.utils.StringHelper

def principal = request.parameters.user
def remoteAddress = request.parameters.remoteAddress
def user = userLocatorHandler.locate(new UserLocatorVO(principal: principal))
def requestData = new RequestData(sessionData.requestData)
if (StringHelper.isNotBlank(remoteAddress)) {
    requestData.remoteAddress = remoteAddress
}
def runAs = SessionDataFactory.direct(user)
        .requestData(requestData)
        .channel(BuiltInChannel.MAIN)
        .build()
def session = sessionHandler.create(new CreateSessionParameters(runAs))
return session.sessionToken

Then create a custom web service, select that script and set the URL mapping to login. When performing a request to <cyclos-root>/run/login?user=consumer1&remoteAddress=183.165.12.7, a session will be created for that user, and the session token will be returned.

It is then possible to redirect the client to <cyclos-root>/?Session-Token=<returned-session-token> and the user will be logged-in to Cyclos.

4.4.14. Service interceptors

These scripts are invoked before and / or after specific service operations. The services are those that extend org.cyclos.services.Service, not the REST api. The REST services use the internal services, so, ultimately, they can be intercepted too.

In order to apply these kinds of scripts, a service interceptor needs to be created, and among its properties, the following can be highlighted:

  • Which service(s) are intercepted;

  • Which operation(s) are intercepted;

  • Which script is executed;

  • Whether the interceptor is enabled or not.

Multiple service interceptors may apply over the same operation. Hence, the order is important. For this reason, interceptors are manually ordered.

Interceptors run in the same database transaction as the regular service operation. Each service operation defines whether the transaction is read-write or read-only. Operations that just read data run in a read-only transaction. In that case, attempting to write data in the database will fail. Also, even if the transaction is read-write, in the script that runs after the operation, it might happen that an error was thrown, marking the transaction as rollback. As such, service interceptor scripts should be very careful when writing to the database.

If the interceptor really needs to write to the database, it is recommended to do it in another database transaction, running after the original transaction ends. The ScriptHelper class (which is bound to the script context on the scriptHelper variable) provides the addOnCommitTransactional and addOnRollbackTransactional methods which allow running a closure after the main transaction ends either as commit or rollback. Those methods run the code block itself inside another transaction, in which it is safe to write to the database.

There is a shared context for all interceptors, of type org.cyclos.impl.system.ServiceInterceptorContext. This context can be used to replace parameters before the original operation invocation, or even to skip the invocation altogether and return a value determined by the script. Also, the context can be used to store attributes which will be shared among interceptors or between the code that runs before and after the service invocation itself. The propertyMissing mechanism from Groovy is supported by the context implementation. So, for example, context.myVariable = 'x' will set the attribute myVariable.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The return value from the script, in both functions that run before or after, is ignored.

Recovering from errors in crucial services

If there is an error in the service interceptor script, and it is applied to crucial services, such as login or the application configuration, it may render the network unusable.

In order to recover from it, it is possible to go to the global mode (<cyclos_root_url>/global), go to the network details and click on "Disable service interceptors". It will disable all service interceptors for that network, allowing the regular usage again.

After fixing the scripts, any interceptors need to be manually enabled again.

Examples
Modifying the general transfers overview default filters

This example will set the default filters on transfer overview to not include chargebacks, nor transfers that were charged back. A service interceptor needs to be applied on the AccountService.getAccountHistoriesOverviewData operation.

The script should have this on the code that runs after the service is executed (the code for before may be left empty):

import org.cyclos.model.banking.accounts.AccountHistoriesOverviewQuery
import org.cyclos.model.banking.transfers.TransferNature

if (context.success) {
    AccountHistoriesOverviewQuery query = context.result.query
    // Include all transfer natures except chargeback
    query.natures = EnumSet.complementOf(EnumSet.of(TransferNature.CHARGEBACK))
    // Also don't include transfers that were themselves charged-back
    query.chargedBack = false
}
Marking mobile phones enabled for SMS by default on registrations by administrators or brokers

This example sets mobile phones to be enabled for SMS by default when registering a user by administrator or broker. To achieve this, create a service interceptor that captures the UserService.getDataForNew operation.

The script should have this on the code that runs after the service is executed (the code for before may be left empty):

if (context.success) {
    def phoneData = context.result?.mobilePhoneData
    if (phoneData?.canManuallyVerify) {
        phoneData.verified = true
    }
}

4.4.15. Custom recurring tasks

These scripts are called periodically by custom recurring tasks. See System – Scheduled tasks for more details.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result
  • String: A message to be stored in the current execution log.

Examples
Periodically importing a file

This example imports a file with users, which is expected to be located at a given directory in the file system. For other import types, it is just a matter of using distinct org.cyclos.model.system.imports.ImportedFileDTO subclasses (some require setting some parameter, like in the example, the group for users). The recurring task just triggers the import. From that point, the import is processed in the background, and the status can be monitored on: System - Tools - Imports menu.

To use it, you will need the following content in the script parameters box (either in the script itself or in the custom recurring task’s script parameters):

filename=/tmp/imports/users.csv
group=consumers

Then use the following code in the script box:

import org.cyclos.model.system.imports.UserImportedFileDTO
import org.cyclos.model.users.groups.GroupVO
import org.cyclos.model.utils.FileSizeUnit
import org.cyclos.server.utils.SerializableInputStream

// Resolve the users filename and the group
String filename = scriptParameters['filename']
String groupInternalName = scriptParameters['group']

// Download the file to a local temp file
File file = new File(filename)
if (!file.exists()) {
    return "The expected file, ${filename}, doesn't exist"
}
if (file.length() == 0) {
    return "The file ${filename} is empty"
}

// Caution! the SerializableInputStream automatically deletes the file
// when closed, except when calling, except when calling .file()
def stream = new SerializableInputStream(file)
stream.file()

// Import
UserImportedFileDTO dto = new UserImportedFileDTO()
dto.fileName = filename
// It is important to mark the file as automatic import,
// otherwise manual interaction would be required for processing
dto.processAutomatically = true
dto.group = new GroupVO([internalName: groupInternalName])
importService.upload(dto, stream, null)

// Build a result string
def fileSize = FileSizeUnit.nearestFileSize(file.length())
return "Started import of ${filename}. File size is ${fileSize}"
Periodically update a static HTML page

In this example, every time the recurring task runs, a static HTML file is updated. In the file, it is written the total number of users and the balances of each system account.

import org.cyclos.entities.users.QUser
import org.cyclos.model.banking.accounts.AccountWithStatusVO
import org.cyclos.model.banking.accounts.SystemAccountOwner
import org.cyclos.model.users.groups.BasicGroupNature
import org.cyclos.model.users.users.UserStatus

import groovy.xml.MarkupBuilder

def now = new Date()

QUser u = QUser.user
int users = entityManagerHandler
        .from(u)
        .where(u.status.notIn(UserStatus.REMOVED, UserStatus.PURGED),
        u.group.nature.in(BasicGroupNature.MEMBER_GROUP, BasicGroupNature.BROKER_GROUP))
        .count()
List<AccountWithStatusVO> accounts = accountService.
        getAccountsSummary(SystemAccountOwner.instance(), null)

File out = new File(scriptParameters.file)

def sessionData = binding.sessionData
def formatter = binding.formatter
MarkupBuilder builder = new MarkupBuilder(new FileWriter(out))
builder.html {
    head {
        title "${sessionData.configuration.applicationName} summary"
        meta charset: "UTF-8"
    }
    body {
        p {
            b "Total users"
            span ": ${users}"
        }
        accounts.each { a ->
            p {
                b a.type.name
                span " balance: ${formatter.format(a.status.balance)}"
            }
        }
        br()
        br()
        br()
        p style: "font-size: small", "Last updated: ${formatter.format(now)}"
    }
}
return "File ${out.absolutePath} updated"

It also needs the script properties to set the file name:

file = /var/www/html/summary.html

4.4.16. Custom background tasks

These scripts are called in a single shot task, which is scheduled to run either immediately or after a given time point.

Whenever a task is scheduled, a string context is stored, and will be later on passed to the script. The script uses this context to determine what to do.

Background tasks are mainly used in 2 use cases:

  • Bulk processing: When many database entities need to be checked / updated in a batch, using a custom recurring task would end up processing all those entities in a single database transaction and in a single CPU. It would probably be much better for that script to just schedule all the background task executions, one per entity to be processed;

  • Long-running task: In cases where a script needs to perform a long-running task, if that task would be triggered by a custom operation or custom web service, for example, it would take too long to return a result. Instead, such an operation can just schedule the execution of the long-running task as a background task, and return immediately.

The custom background task execution is logged following the same semantics as the built-in background tasks. The global configuration has a setting for background tasks logging, which can be all, default or verbose. When configuring the background task in the System > Tools > Scheduled tasks > Background tasks tab, there’s a checkbox indicating whether successful executions of the task should be handled as verbose. If so, successful executions will only be logged to the global-tasks log if the global configuration setting is to log verbose tasks.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script result is ignored.

Scheduling background tasks

Custom background tasks are always scheduled to run from other scripts. For this, use the org.cyclos.impl.utils.tasks.BackgroundTaskHandler's custom() method.

Its arguments are the custom background tasks' internal name and the context. Don’t forget to call the schedule() method in the result. See the examples section.

Using the fork-join model

The fork-join model consists of submitting a number of parallel tasks for execution, and then resuming execution when they have all finished.

Starting with Cyclos 4.16, it is possible to apply this model to background tasks. An example built-in feature that uses this model is the account fee charges. When dispatching an account fee charge, Cyclos schedules a background task execution for each 200 users that should be charged. And after all those are finished, the execution is marked as finished (and notified).

For custom background tasks to use a fork-join, at the moment of the scheduling the number of tasks must be known beforehand. A call to def forkJoin = backgroundTaskHandler.newForkJoin(n, 'code'[, …​libraryInternalNames]) is needed, being n the number of tasks and code the groovy code that will be submitted for execution after all tasks finish. The newForkJoin method also receives a variable number of internal names for library scripts, which will be available when running the specified code. Also note that if you have each background task to process a batch, not a single record, you need to calculate the number of tasks as: n = Math.ceil(count * 1.0 / batchSize) as int.

The script code is executed at a later time, when all tasks have finished. Please, note that there’s no guarantee that the join code will be executed immediately after the execution of all background tasks, but it can take up to a minute for this to happen. Anyway, this feature is really useful when processing a lot of data, so this extra time will be barely noticeable.

Try to keep the join code as small as possible, because it will be harder to debug the code which is passed in dynamically in a string. Instead, create a library and just call a function in that library passing some form of dynamically generated id. See the example below for more details.

Examples
Scheduling a bulk of custom background tasks

This script is used to schedule custom background tasks. Its type is Custom recurring task, but any kind of script could schedule background tasks.

import org.cyclos.impl.utils.QueryHelper
import org.cyclos.model.users.records.UserRecordQuery
import org.cyclos.model.users.recordtypes.RecordTypeVO

// Retrieve the ids to process
def ids = recordSearchHandler.iterateIds(new UserRecordQuery(
        type: new RecordTypeVO(internalName: 'recordInternalName')))

// For each one, schedule a background task which will process it
QueryHelper.processBatch(entityManagerHandler, ids) { Long id ->
    def context = id.toString()
    backgroundTaskHandler.custom("taskInternalName", context).schedule()
}
Scheduling a bulk of custom background tasks with fork-join

This is basically the same script as in the previous example. However, it uses a fork-join, assuming a library script exist with internal name 'libraryInternalName', and defines a function to notify that all records have been processed the execution has finished.

Library that defines the functions to start and finish the executions:

def newExecution() {
    def id = UUID.randomUUID()
    println "Starting execution of $id"
    return id
}

def finishExecution(id) {
    println "Finishing execution of $id"
}

Recurring task that actually schedules the background tasks using a fork-join:

import org.cyclos.model.users.records.UserRecordQuery
import org.cyclos.model.users.recordtypes.RecordTypeVO

// Library function that creates a new execution and returns the id
def execution = newExecution()

// Get the record ids to process
def ids = recordSearchHandler.iterateIds(new UserRecordQuery(
        type: new RecordTypeVO(internalName: 'recordInternalName')))
        .toList()

if (ids.empty) {
    // Nothing to process!
    return 'Nothing to process'
}

// Create a new fork-join instance, calling the code that
// finishes the execution, and including the library.
// Take care to property wrap strings between quotes.
def forkJoin = backgroundTaskHandler.newForkJoin(ids.size(), """
    finishExecution('${execution}')
""", 'libraryInternalName')

// For each one, schedule a background task which will process it
ids.each { Long id ->
    def context = id.toString()
    // Pass the fork-join id to the 'custom' method
    backgroundTaskHandler.custom('taskInternalName', context, forkJoin)
            .schedule()
}

return "Scheduled ${ids.size()} tasks"
Process each record

In both the previous examples, many background tasks are scheduled, one per record id. This example processes each one.

// Use the context as record id
def record = recordService.find(Long.parseLong(context))
def fields = scriptHelper.wrap(record)
if (fields.status?.internalName == 'toProcess') {
    // Do some processing with this record, then mark it as done
    fields.status = 'done'
}

4.4.17. Custom SMS operations

These scripts are invoked when a user executes a custom sms operation, as set in the sms channel in the configuration. The function should implement the logic for that operation.

In terms of permissions, the same concerns applied to custom web services should also be applied to custom SMS operations as well, because there is no built-in security layer.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result

The script result is ignored.

Examples
Pay taxi with an SMS message

In this example SMS operation, users can pay taxi drivers via SMS. Make sure all the following are configured:

  • In the script details, the checkbox "Run with all permissions" is disabled;

  • There should be a single transfer type enabled for the SMS operations channel, and the user performing the operation needs to have permission to perform that payment;

  • A custom profile field with internal name taxiId of type single line text, and marked as unique needs to be enabled for the product of taxi owners;

  • A user identification method of type custom field, called "Taxi id" with the taxiId field needs to be created. Make sure its internal name is also taxiId;

  • In the configuration details, in the channels tab, enable SMS operations. Then, in that channel, make sure "Taxi id" is allowed as user identification method to perform payments;

  • Still in the same channel configuration page, create a new SMS operation of type Custom, selecting the alias taxi and the selected script.

Then, customers can perform the payment by sending a sms in the format: taxi <taxi id> <amount>. Below is the script that should be used:

import org.cyclos.model.banking.TransferException
import org.cyclos.model.banking.transactions.PerformPaymentDTO
import org.cyclos.model.banking.transfertypes.TransferTypeVO
import org.cyclos.model.messaging.sms.OutboundSmsType
import org.cyclos.model.users.users.UserLocatorVO

// Read the parameters
String taxiId = parameterProcessor.nextString("taxiId")
BigDecimal amount = parameterProcessor.nextDecimal("amount")

// Find the user by taxi id
def locator = new UserLocatorVO(
        principalType: "taxiId",
        principal: taxiId)

// Perform the payment
def pmt = new PerformPaymentDTO()
pmt.amount = amount
pmt.owner = phone.user
pmt.subject = locator
pmt.type = new TransferTypeVO(internalName: scriptParameters.paymentType)
try {
    vo = paymentServiceSecurity.perform(pmt)
    outboundSmsHandler.send(phone,
            "The payment was successful",
            OutboundSmsType.SMS_OPERATION_RESPONSE)
    // Also notify the taxi, for example, by connecting to the
    // taxi company system, which notifies the taxi driver...
} catch (TransferException e) {
    outboundSmsHandler.send(phone,
            "The payment couldn't be performed",
            OutboundSmsType.SMS_OPERATION_RESPONSE)
}

Also, set in the script parameters the payment type to be used, with the format accountTypeInternalName.transferTypeInternalName:

paymentType = userUnits.taxiPayment

4.4.18. Inbound SMS handling

These scripts are invoked when a gateway sends SMS messages to Cyclos. There are two functions in this script: one to generate the gateway response and another one to resolve basic SMS data from an inbound HTTP request. Both functions are optional, defaulting to the normal behavior (when not using a script).

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Resolve basic SMS data

This function is used to read an inbound sms request and return an object containing the phone number, the SMS message and the split SMS message into parts. Only the phone number and SMS message are required. If the message parts are empty, it will be assumed the message will be split by spaces.

Result
Return response to gateway

This function is used to determine the HTTP status code, headers and body to be returned to the SMS gateway. It can be called either when the bare minimum parameters – mobile phone number and sms message – were not sent by the gateway or when the gateway has sent a valid SMS. Keep in mind that if an operation has resulted in error, from a gateway perspective, the SMS was still delivered correctly, and the response should be a successful one. Maybe when the bare minimum parameters weren’t sent, the script could choose to return a different message. When no code is given, the default processing will be done, returning the HTTP status code 200 with "OK" in the body.

Additional bound variables
Script result
Examples
Receiving an SMS in JSON format

This example assumes the request body is a JSON object:

import java.nio.charset.StandardCharsets

import org.cyclos.impl.utils.sms.InboundSmsBasicData

def body = new InputStreamReader(request.body, StandardCharsets.UTF_8)
def json = objectMapper.readTree(body)

def result = new InboundSmsBasicData()
result.phoneNumber = json.get("phoneNumber")?.asText()
result.message = json.get("message")?.asText()
return result
Receiving an SMS with a custom format

This example reads the phone number from a request header, and the message from the request body:

import org.apache.commons.io.IOUtils
import org.cyclos.impl.utils.sms.InboundSmsBasicData

// Read the phone from a header, and the message from the body
def result = new InboundSmsBasicData()
result.phoneNumber = request.headers."phone-number"
println(request.headers)
result.message = IOUtils.toString(request.body, "UTF-8")
println(result.message)
return result

4.4.19. Outbound SMS handling

These scripts are invoked to send SMS messages. By default, Cyclos connects to gateways via HTTP POST / GET, which can be set in the configuration. However, the sending can be customized (or totally replaced) via a script.

As in most cases the custom sending just wants to customize some aspects of the sending, not all, it is possible that the script just creates a subclass of org.cyclos.impl.utils.sms.GatewaySmsSender, customizing some aspects of it (for example, by overriding the buildRequest method and adding some headers, or the resolveVariables method to have some additional variables which can be sent in the POST body).

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result
Examples
Sending SMS requests as JSON

This example posts the SMS message as JSON to the gateway, and awaits the response before returning the status:

import java.nio.charset.StandardCharsets

import org.cyclos.model.messaging.sms.OutboundSmsStatus
import org.springframework.http.HttpEntity
import org.springframework.http.HttpHeaders
import org.springframework.http.MediaType

// Read some gateway data from the configuration
def smsConfig = configuration.outboundSmsConfiguration
def url = smsConfig.gatewayUrl
def user = smsConfig.username
def pwd = smsConfig.password

// Prepare the request headers
def headers = new HttpHeaders()
headers.setContentType(MediaType.APPLICATION_JSON)
if (user) {
    headers.setBasicAuth(user, pwd, StandardCharsets.UTF_8)
}
// Build the JSON body
def body = [
    to: phoneNumber,
    text: message
]
// Send the request
try {
    rest.postForObject(url, new HttpEntity(body, headers), HttpEntity)
    return OutboundSmsStatus.SUCCESS
} catch (Exception e) {
    return OutboundSmsStatus.UNKNOWN_ERROR
}
Sending SMS requests as XML

This example posts the SMS message as XML to the gateway, and awaits the response before returning the status:

import java.nio.charset.StandardCharsets

import org.cyclos.model.messaging.sms.OutboundSmsStatus
import org.springframework.http.HttpEntity
import org.springframework.http.HttpHeaders
import org.springframework.http.MediaType

import groovy.xml.MarkupBuilder

// Read some gateway data from the configuration
def smsConfig = configuration.outboundSmsConfiguration
def url = smsConfig.gatewayUrl
def user = smsConfig.username
def pwd = smsConfig.password

// Prepare the request headers
def headers = new HttpHeaders()
headers.setContentType(MediaType.APPLICATION_XML)
if (user) {
    headers.setBasicAuth(user, pwd, StandardCharsets.UTF_8)
}
// Build the XML body
def body = new StringWriter()
new MarkupBuilder(body)."sms-message" {
    "destination-phone" phoneNumber
    text message
}
// Send the request
try {
    rest.postForObject(url, new HttpEntity(body.toString(), headers), HttpEntity)
    return OutboundSmsStatus.SUCCESS
} catch (Exception e) {
    return OutboundSmsStatus.UNKNOWN_ERROR
}

These scripts are used to generate links (URLs) which are used to point users to specific functionality. Some systems have a custom front-end for users, which means that when they receive e-mails with links, instead of pointing the links to the default Cyclos page, it should point to the custom front-end page.

Whenever the script returns null, the default link to Cyclos is generated, so the script may handle specific users / groups, and fallback to the default for other users by returning null.

To actually use a custom link generation script, it has to be set in the Cyclos configuration in System > System configuration > Configurations.

The script code has the following bound variables (besides the default bindings):

  • type: The org.cyclos.impl.utils.LinkType to be generated;

  • user: The org.cyclos.entities.users.BasicUser for which the link is being created. Can be null in some cases;

  • urlFilePart: The URL part which is used by the default link in Cyclos. Kept mostly for backwards compatibility, because if the default is desired, the script should return null.

  • String: The URL used for the link;

  • Null: Returning null indicates the default link is used.

Generate a link to the login page. The type variable is LOGIN.

No additional variables for this type.

Generate a link to the root page. The type variable is ROOT.

No additional variables for this type.

Generate a link to the home page. The type variable is HOME.

No additional variables for this type.

Generate a link to a specific built-in location, optionally passing a parameter. The type variable is NOTIFICATION.

  • location: The org.cyclos.model.utils.Location;

  • entityId: The identifier of the entity related to the notification;

  • entityIdParam: The parameter name to pass the entity identifier.

Generate a link which validates a user registration. The type variable is REGISTRATION_VALIDATION.

  • validationKey: The key which is sent by e-mail to validate the action.

Generate a link to a custom operation callback URL. The type variable is EXTERNAL_REDIRECT.

Generate a link to a custom operation callback URL. The type variable is WIZARD_EXTERNAL_REDIRECT.

Generate a link to resume a wizard execution.

Generate a link to pay a ticket, which is another, simplified application provided by Cyclos. The type variable is TICKET.

  • ticket: The org.cyclos.entities.banking.Ticket to be paid. This class contains the 'ticketNumber' which is used to pay the ticket, and hence, should be appended to the generated URL.

Generate a link to pay an easy invoice, which is another, simplified page provided by Cyclos. The type variable is EASY_INVOICE.

Generate a URL with a custom scheme pointing to a mobile application page (e.g. cyclos://history?id=euros_account). The type variable is MOBILE.

If you configured the mobile application to use a URL scheme different than the default (cyclos) then, you need to handle this link type to return the URL accordingly. See the example below.

  • mobileUrlFilePart: The URL part with the mobile page and parameters (if any). This value doesn’t contain a leading / (e.g. history?id=euros_account). To know the list of available pages please check the mobile application reference documentation.

NOTE: Most likely what you need to customize is for the type MOBILE, see above.

Generate a link to Cyclos that in turn will redirect to a mobile application page (running the link generation script with type MOBILE, see above). The type variable is MOBILE_REDIRECT.

Rarely will you need to return something different for this type, the default link should be enough for all cases: <root_url>/mobile-redirect/<mobileUrlFilePart>.

  • mobileUrlFilePart: The URL part with the mobile page and parameters (if any). This value doesn’t contain a leading /.

Generate a link to directly reply to an internal message. The type variable is REPLY_MESSAGE.

Generate a link to a simplified page which allows users to unsubscribe from emails of a given type (direct message, notifications, email mailings, etc.). The type variable is EMAIL_UNSUBSCRIBE.

Generate a link which validates a user email change. The type variable is EMAIL_CHANGE.

  • validationKey: The key which is sent by e-mail to validate the action.

Generate a link to the simplified page that shows details of a voucher. The type variable is VOUCHER_INFO.

  • voucher: The voucher to which the link will be generated, as org.cyclos.entities.banking.Voucher. Maybe null, in which case should point to a page where the user can type-in the voucher token.

Generate a link to the URL that will redirect to the identity provider, when using the 'Login with' Google, Facebook, Microsoft, etc. The type variable is IDENTITY_PROVIDER_REDIRECT.

Generate a link to the URL which handles callback URLs passed to identity providers. The type variable is IDENTITY_PROVIDER_CALLBACK.

Generate a link for a user invitation.

This example generates the links to the frontend when hosted separately from Cyclos. To use it, you will need the following content in the script parameters box:

rootUrl = https://account.example.com

Then, use the following script code:

import org.cyclos.entities.users.BasicUser
import org.cyclos.impl.utils.LinkType
import org.cyclos.utils.StringHelper

BasicUser user = binding.user
if (user?.admin && user.user.group.adminType != null) {
    // Don't generate custom links for system administrators
    return null
}

// Read the parameters
Map scriptParameters = binding.scriptParameters
LinkType linkType = binding.type
String root = StringHelper.removeEnd(scriptParameters.rootUrl, '/')

// For root, return the configured root URL
if (linkType == linkType.ROOT) {
    return root
}

// Cyclos already generates links to the built-in frontend,
// using the /ui/ prefix. This script assumes that the users
// configuration sets the new frontend for all regular users.
String urlFilePart = binding.urlFilePart
if (urlFilePart?.startsWith("/ui/")) {
    return root + StringHelper.removeStart(urlFilePart, "/ui")
}

This example generates links to mobile application pages using a custom URL scheme.

For this to work as expected and open your mobile application when clicking on a link, you must have configured the app with this same scheme. Please check the mobile application reference documentation to know how to do it.

import org.cyclos.impl.utils.LinkType

// For link types different than 'MOBILE', return null to generate the default links
if (type != LinkType.MOBILE) {
    return null
}

// Return a link using the configured 'myApp' URL scheme for the mobile application
// The only thing we must do is to concatenate the custom URL scheme at the beginning,
// leaving the parameter unchanged
return "myApp://${mobileUrlFilePart}"

4.4.21. Phone number handling

Cyclos uses Google’s excellent libphonenumber library to handle phone numbers. It is able to perform many tasks related to phone numbers, given a default country (numbers may include the country code itself). Some examples include validating a phone number, determining the number type (mobile, fixed line or both), formatting a phone number (national, international and E.164 formats), generating example numbers, and much more.

Phone number rules across countries change over time. In this case, it usually takes a few weeks for a new release of libphonenumber to cover the new rules. It takes more time for a new Cyclos version, with the new library version, to be released. If your project is affected by a number rule change, it is always advisable to report a bug in libphonenumber, and manually update the library in your Cyclos installation as soon as it is released (by replacing the WEB-INF/lib/libphonenumber-<version>.jar with the new one, making sure only one version exists).

However, for projects that cannot wait a new release of libphonenumber, starting with Cyclos 4.16, a new kind of script was introduced: phone number handling.

To actually use a phone number handling script, it has to be set in the Cyclos configuration in System > System configuration > Configurations.

The script has 2 blocks:

Parse

This script function is called when parsing a phone number entered by some user. The script is responsible for parsing, validating, determining the number type and formatting the phone number. If null is returned, the regular parsing by libphonenumber will be performed.

Additional bound variables

The parse function has the following bound variables (besides the default bindings):

  • number: The raw phone number, as typed-in by the user;

  • country: The default ISO-3166 2-letter country code. The number may include a country itself by starting with +country_code.

Script result

The script may return one of the following:

  • Null: When returning null, the default parsing will be performed, with libphonenumber;

  • False: Returning false indicates that the number is invalid, and the regular parsing won’t be attempted;

  • Object / Map: Must be an object compatible with org.cyclos.impl.utils.PhoneNumberData. Basically, need the following properties:

    • e164: String, required. The E.164 formatted number from the given input number;

    • mobile: Boolean, required. Indicates whether the given input number can be used as mobile number;

    • landLine: Boolean, required. Indicates whether the given input number can be used as land-line (fixed) number. Some countries / rules might allow a phone to be used as both types;

    • international: String, optional. The international representation of the input number. When not set, the E.164 format is used.

    • national: String, optional. The national representation of the input number. When not set, the international or E.164 format is used.

Generate an example number

This script function is called when an example number should be displayed for users.

Additional bound variables

The example number generation function has the following bound variables (besides the default bindings):

  • mobile: Flag indicating whether to return a mobile phone number example;

  • landLine: Flag indicating whether to return a (fixed) land-line phone number example;

  • country: The ISO-3166 2-letter country code.

Script result
  • Null: When returning null, the default example from libphonenumber will be used;

  • String: The example number.

Examples
Custom Brazilian phone number handling

This example does custom handling of Brazilian phone numbers (country dialing code is 55). Brazilian numbers are fully supported by libphonenumber, but this is just a didactic example.

Function to parse numbers:

import org.cyclos.utils.StringHelper

def numbers = StringHelper.numbersOnly(number)
def hasCountryCode = numbers.startsWith("55");

if (!hasCountryCode && country != 'BR') {
    return null
}

if (hasCountryCode) {
    numbers = numbers.substring(2)
}
def landline = numbers.length() == 10
def mobile = numbers.length() == 11

if (!landline && !mobile) {
    return false
}

def areaCode = numbers.substring(0, 2)
def firstPart = numbers.substring(2, mobile ? 7 : 6)

def mobilePrefix = firstPart.startsWith("9") || firstPart.startsWith("8")
if (mobile && !mobilePrefix || landline && mobilePrefix) {
    return false
}

def secondPart = numbers.substring(mobile ? 7 : 6, numbers.length())
def local = "${firstPart}-${secondPart}"

return [
    landLine: landline,
    mobile: mobile,
    e164: "+55${numbers}",
    national: "(${areaCode}) ${local}",
    international: "55 ${areaCode} ${local}"
]

Function to generate example numbers:

if (country != 'BR') {
    return null
}
return mobile ? "01 90123-4567" : "01 3012-3456"

4.4.22. IP geolocation

These scripts are used to map an IP address to a geolocation. This is useful in different contexts. For example, users may be notified when accessing their accounts from a new device, showing the approximate location. Or the IP address list can also show the approximate location.

To actually use an IP geolocation script, it has to be set in the Cyclos configuration in System > System configuration > Configurations.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

  • address: The IP address, as string.

Script result
  • org.cyclos.entities.system.IpGeolocation (or a compatible Map): the geolocation result. Neither address nor expirationDate fields need to be returned - they will always be overridden;

  • Null: When not returning anything, it will be considered that the address cannot be located.

Examples
IP geolocation with IPinfo.io

This example uses https://ipinfo.com/. IPinfo.io allows free usage for up to 50k requests per month, allowing paid plans with higher limits. You need to sign in to get an API key (which they call token). That API key should be set in the script parameters with apiKey = <your-api-key>. Then, paste the following code in the script block:

import org.cyclos.entities.system.IpGeolocation
import org.cyclos.entities.utils.LatLong
import org.springframework.http.HttpEntity
import org.springframework.http.HttpHeaders
import org.springframework.http.HttpMethod
import org.springframework.web.client.RestTemplate

Map<String, String> scriptParameters = binding.scriptParameters
String address = binding.address
RestTemplate rest = binding.rest
def url = "https://ipinfo.io/${address}"

def headers = new HttpHeaders()
headers.add('Authorization', "Bearer ${scriptParameters.apiKey}")
def result = rest.exchange(url, HttpMethod.GET,
        new HttpEntity(headers), Map)

def map = result.body
def loc = map.loc.split(',')
return new IpGeolocation(
        country: map.country,
        region: map.region,
        city: map.city,
        location: new LatLong(loc[0] as BigDecimal, loc[1] as BigDecimal))
IP geolocation with ipbase

This other example uses https://ipbase.com/. Its free usage allows only up to 150 requests per month, but paid plans are reasonably cheap. You need to sign in to get an API key. That API key should be set in the script parameters with apiKey = <your-api-key>. Then, paste the following code in the script block:

import org.cyclos.entities.system.IpGeolocation
import org.cyclos.entities.utils.LatLong
import org.springframework.http.HttpEntity
import org.springframework.http.HttpHeaders
import org.springframework.http.HttpMethod
import org.springframework.http.HttpStatus
import org.springframework.web.client.HttpServerErrorException
import org.springframework.web.client.RestTemplate

Map<String, String> scriptParameters = binding.scriptParameters
String address = binding.address
RestTemplate rest = binding.rest
def url = "https://api.ipbase.com/v2/info/?ip=${address}"

def headers = new HttpHeaders()
headers.add('apiKey', scriptParameters.apiKey)
def result
try {
    result = rest.exchange(url, HttpMethod.GET,
            new HttpEntity(headers), Map)
} catch (HttpServerErrorException e) {
    if (e.statusCode == HttpStatus.NOT_FOUND) {
        // The IP was not found in IpBase
        return null
    }
    throw e
}

def location = result.body.data.location
return new IpGeolocation(
        country: location.country.alpha2,
        region: location.region.name,
        city: location.city.name,
        location: new LatLong(location.latitude, location.longitude))

4.4.23. Export formats

These scripts are invoked when exporting data to a file in a custom format.

There are several contexts which can be exported:

  • Account history;

  • Transfers overview;

  • Transactions search (such as scheduled payments search);

  • Transactions overview (such as payment requests overview);

  • Payment details (such as payment, scheduled payment, payment request, external payment or transfer);

  • Users search;

  • User balances overview;

  • Account limits overview;

  • Records search (for system or specific user, of a given type);

  • Records overview (as administrator or broker, of a given type);

  • Shared fields records search;

  • Tokens search (such as cards);

  • Vouchers search;

  • Voucher details;

  • Custom operation results (when returning a result page).

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result
  • java.io.InputStream: The binary file content;

  • byte[]: The binary file content;

  • java.io.Reader: The textual file content;

  • java.io.File: The file to read the content. The file will be deleted after the content is read;

  • Otherwise, it will call the toString() method on the result and assume a textual file content.

Examples
Exporting the account history as Swift MT940 format

This script allows exporting the account history entries in the MT940, which is used by some accounting software for importing / exporting transactions:

First create a script of type Export format with the following code:

import java.text.SimpleDateFormat

import org.cyclos.entities.banking.Account
import org.cyclos.entities.users.User
import org.cyclos.impl.banking.AccountHistoryEntry
import org.cyclos.model.banking.accounts.AccountHistoryQuery
import org.cyclos.utils.StringHelper

def timeZone = sessionData.configuration.timeZone
def dateFormat = new SimpleDateFormat("yyMMdd")
dateFormat.timeZone = timeZone
def entryDateFormat = new SimpleDateFormat("yyMMdd")
entryDateFormat.timeZone = timeZone

def formatAmount(BigDecimal amount) {
    return amount.abs().toPlainString().replace('.', ',')
}

def formatSignal(BigDecimal amount) {
    return amount.compareTo(BigDecimal.ZERO) > 0 ? 'C' : 'D'
}

def formatOwner(Account account) {
    String text
    if (account.owner instanceof User) {
        text = account.owner.username
    } else {
        text = account.type.internalName ?: account.type.name
    }
    return formatText(text)
}

def formatDescription(AccountHistoryEntry entry) {
    def description = entry.transaction?.description ?: entry.type.valueForEmptyDescription
    return formatText(description)
}

def formatText(text) {
    // First replace line breaks or multiple spaces by a single space, trimming to 60 chars
    text = (text ?: '').replaceAll("[\n|\r]+", " ")
    text = text.replaceAll("\\s+", " ")
    text = StringHelper.trim(StringHelper.truncate(text, 60))
    // Second make sure that no special characters are used
    text = StringHelper.asciiOnly(StringHelper.unaccent(text))
    // Finally make sure that no colon character is used, this might mess up the mt940 file
    return text.replaceAll('\\:', ' ')
}

// Get the account
AccountHistoryQuery query = binding.query
Account account = conversionHandler.convert(Account, query.account)

// Get the begin date
Date begin = conversionHandler.toDate(query.period?.begin) ?: account.creationDate

// Get the end date
Date now = new Date()
Date end = conversionHandler.toDate(query.period?.end) ?: now
if (end.after(now)) {
    end = now
}

// Get the balance at begin / end
def balanceBegin = accountService.getBalance(account, begin)
def balanceEnd = accountService.getBalance(account, end)
def currency = scriptParameters.currencyCode

// Write the header
StringBuilder out = new StringBuilder(""":20:CN${dateFormat.format(end)}
:25:${scriptParameters.iban}
:28:000
:60F:${formatSignal(balanceBegin)}${dateFormat.format(begin)}${currency}${formatAmount(balanceBegin)}
""")

// Process each entry
scriptHelper.processBatch(data) { AccountHistoryEntry entry ->
    def date = entryDateFormat.format(entry.date)
    def amount = formatAmount(entry.amount)
    def signal = formatSignal(entry.amount)
    def fromTo = formatOwner(entry.relatedAccount)
    def description = formatDescription(entry)
    out << ":61:${date}${signal}${amount}NOV NONREF\n"
    out << ":86:${fromTo} > ${description}\n"
}

// Write the footer
out << ":62F:${formatSignal(balanceEnd)}${dateFormat.format(end)}${currency}${formatAmount(balanceEnd)}"

// Return the output content
return out

Also set the following in the script parameters box:

# The currency code that will be exported on the file
currencyCode = EUR
# The IBAN account number that will be exported in the file
iban = NL70TRIO0123456789

Then, create a new export format in System > System configuration > Export formats, with the following fields:

  • Name: MT940 (change as desired);

  • Internal name: mt940;

  • Content type: application/octet-stream;

  • Binary: No;

  • Character encoding: UTF-8;

  • File extension: mt940;

  • Contexts: Account history;

  • Script: Select the previously created script.

4.4.24. Notifications

These scripts are invoked before generating the notification to be stored in the database. Later, the notifications will be sent through a background task.

For the script to be actually used, it needs to be set in the Cyclos configuration.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Script result
  • Null: No customizations are made for this notification;

  • A Map with the following properties, each could be null to fall back to the default notification:

    • title: The email notification title, as string;

    • body: The email notification body, as string;

    • sms: The SMS message, as string;

    • fcm: Allows customizing push notifications sent via Firebase Cloud Messaging. Is another map with:

      • title: The push notification title. If not given, then the title above will be used (if any);

      • body: The push notification body. If not given, then the body above will be used (if any);

      • imageUrl: The url of the image associated with the push notification. If not given, the user’s profile image is used (only for users, not operators);

      • iosBadge: A boolean flag indicating if a badge must be shown for iOS notifications. Default is true;

      • androidIconColor: The color in #rrggbb format (e.g: #23AB34) used to colorize the small notification icon shown in Android devices. By default, no color is sent. Device support depends on the Android version;

      • data: A Map<String, String> used to send additional data to the mobile application.

Examples
Include the balance for a "Payment received" notification

This script will add the user balance to the body of the notification generated for the type: PAYMENT_RECEIVED:

import org.cyclos.model.messaging.notifications.AccountNotificationType
import org.cyclos.utils.StringHelper

def received = type == AccountNotificationType.PAYMENT_RECEIVED
def performed = type == AccountNotificationType.ALL_NON_SMS_PERFORMED_PAYMENTS
if (received || performed) {
    def account = received ? entity.to: entity.from
    def balance = formatter.format(entity.currency,
            accountService.getBalance(account, null))
    def amount = formatter.format(entity.currencyAmount)
    def owner = formatter.format(received ? entity.fromOwner: entity.toOwner)
    def ownerShort = StringHelper.truncate(owner, 30)
    if (received) {
        return [
            body: "You have received a payment of $amount from $owner."
            + " Your new balance is: $balance.",
            sms: "Payment of $amount received from $ownerShort."
            + " New balance: $balance."
        ]
    } else {
        return [
            body: "You have performed a payment of $amount to $owner."
            + " Your new balance is: $balance.",
            sms: "Payment of $amount performed to $ownerShort."
            + " New balance: $balance."
        ]
    }
}

// default values for the rest of the notification types
return null

4.4.25. Content helper

These scripts are used to generate data which is passed in to the Thymeleaf context when processing dynamic content.

Additional bound variables

The script code has the following bound variables (besides the default bindings):

Examples
Show user records in a content

This script adds the current user’s records of a hard-coded type to be later on processed by Thymeleaf. Assuming there’s a record type with internal name remark which has a field with internal name description, this example fetches and wraps each record, so the template can easily use it:

import org.cyclos.entities.users.UserRecordType

def type = entityManagerHandler.find(UserRecordType, 'recordType')
def records = recordService.listUser(type, sessionData.loggedUser)

return [
    records: records.collect {
        scriptHelper.wrap(it)
    }
]

And this is an example content that shows the description of each record, together with its creation date:

<div th:each="remark: ${records}" style="display: flex">
    <div th:text="${#format.object(remark.creationDate)}">Date</div>
    <div>&nbsp;</div>
    <div th:text="${remark.description}">Description</div>
</div>

4.4.26. Running scripts directly

In many cases, it is handy for administrators to run scripts directly. So, instead of having to create a custom operation script, then a custom operation, then granting permissions, refreshing the browser and running, there is a menu called Run script, which presents a text box where the script may be typed in or pasted, which can be executed directly. Of course, only the default bindings are available.

Result
  • String: The text is displayed as plain text;

  • org.cyclos.model.system.scripts.ScriptResult: This is the way to return either a notification by setting the notification property as string or an HTML-formatted text, by setting the richText property;

  • Map: A Map compatible with ScriptResult will be handled in the same way as it;

  • File, InputStream or Reader: used to return a file;

  • org.cyclos.model.utils.FileInfo: return a file, with more control over it.

So, for example, to return an HTML text with a title, the script can return [title:"The result title", richText:"<b>Formatted</b> text"]. To show a notification, the script can return [notification:"Notification text"]. The same prefixes available on notifications for custom operations are available on notifications: [INFO], [WARN] and [ERROR].

Examples
Remove all users, transactions and related data

Here is an example of a script to remove all regular users (not administrators) and related data, as well as all system to system transactions. This script must be executed in a network. Be advised that there will be no confirmation, and all users and all related data will be removed.

The script works by first recreating all database constraints with the option ON DELETE CASCADE. Then, all users are removed, which will cascade the removal to accounts, transfers, advertisements, records, messages, notifications, references and so on. For this reason, all tables are locked, and the script will likely fail if there is any activity in the system, such as active users or background tasks. If this script doesn’t work because some tables are locked, run the command-line application instead.

Be careful when running in systems where specific users are used in the configuration, such as fees that are paid by a specific user, or payment types which are restricted to specific users. In such configurations, all such related data will be removed as well. Also, note that it may take a while to run, so, please, wait before the script completes.

AGAIN: be very careful when using this script! Only run it on test instances and always have a database backup before running it.

import org.cyclos.db.DeleteNetworkData
import org.cyclos.impl.utils.cache.CacheType
import org.cyclos.model.ValidationException

if (sessionData.network == null) {
    throw new ValidationException("This script can only be executed in a network")
}

def deleteNetworkData = beanHandler.autowire(DeleteNetworkData)
def users = deleteNetworkData.deleteUsersAndBanking(sessionData.network)
CacheType.all().each { cacheHandler.scheduleClear(it) }
searchHandler.reindex()
return "Removed ${users} users"
Export advertisements with images

Here is an example of a script to export the published advertisements with its images in the same format used in the Cyclos import, so the result can be used to add the returned advertisements in other Cyclos networks.

import java.nio.charset.StandardCharsets
import java.sql.ResultSet
import java.util.zip.ZipEntry
import java.util.zip.ZipOutputStream

import org.apache.commons.io.IOUtils
import org.apache.commons.lang3.StringUtils
import org.cyclos.entities.marketplace.AdImage
import org.cyclos.entities.marketplace.QAdImage
import org.cyclos.impl.access.SessionData
import org.cyclos.impl.marketplace.AdImageServiceLocal
import org.cyclos.impl.storage.StoredFileHandler
import org.cyclos.impl.utils.formatting.FormatterImpl
import org.cyclos.impl.utils.persistence.EntityManagerHandler
import org.cyclos.model.marketplace.advertisements.BasicAdVO
import org.cyclos.model.utils.FileInfo
import org.cyclos.server.utils.SerializableInputStream
import org.cyclos.utils.ContentType
import org.springframework.jdbc.core.ColumnMapRowMapper
import org.springframework.jdbc.core.JdbcTemplate
import org.springframework.jdbc.core.RowCallbackHandler

import com.opencsv.CSVParserBuilder
import com.opencsv.CSVWriterBuilder

JdbcTemplate jdbc = binding.jdbc
SessionData sessionData = binding.sessionData
EntityManagerHandler entityManagerHandler = binding.entityManagerHandler
StoredFileHandler storedFileHandler = binding.storedFileHandler
FormatterImpl formatter = binding.formatter
AdImageServiceLocal adImageService = binding.adImageService

def sql = """
select
    ad.id,
    u.username as user,
    ad.creation_date as creationdate,
    ad.name as title,
    ad.description,
    ad.status,
    array_to_string(array_agg(
        cy_name_hierarchy(ac.category_id, 'ad_categories', 'internal_name')
    ), ',') as categories,
    ad.begin_publication_period as publicationbegin,
    ad.end_publication_period as publicationend,
    ad.price_amount as price,
    ad.promotional_price as promotionalprice,
    ad.begin_promotional_price_period as promotionalperiodbegin,
    ad.end_promotional_price_period as promotionalperiodend
from ads ad
    inner join users u on ad.owner_id = u.id
    inner join ads_categories ac on ad.id = ac.ad_id
    inner join ad_categories c on ac.category_id = c.id
where u.network_id = ${sessionData.network.id}
and ad.end_publication_period > now()
group by 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12
"""

def file = File.createTempFile("export", ".zip")
def zip = new ZipOutputStream(new FileOutputStream(file), StandardCharsets.UTF_8)

// Write the index
zip.putNextEntry(new ZipEntry("index.csv"))
def parser = new CSVParserBuilder().withSeparator(sessionData.configuration.listSeparator.value as char).build()
def csv = new CSVWriterBuilder(new OutputStreamWriter(zip, StandardCharsets.UTF_8)).withParser(parser).build()
csv.writeNext([
    'user',
    'creationdate',
    'title',
    'description',
    'status',
    'categories',
    'publicationbegin',
    'publicationend',
    'price',
    'promotionalprice',
    'promotionalperiodbegin',
    'promotionalperiodend',
    'images'
] as String[])
def mapper = new ColumnMapRowMapper()
def rch = { ResultSet rs ->
    def map = mapper.mapRow(rs, 0)
    def images = adImageService.list(new BasicAdVO(map.id as long))
    def imageNames = images.collect {
        def contentType = ContentType.getByMimeType(it.contentType)
        return "${it.id}.${contentType.extension}"
    }
    csv.writeNext([
        map.user,
        formatter.format(map.creationdate),
        map.title,
        map.description,
        map.status,
        (map.categories as String).tokenize(',').collect {StringUtils.strip(it, '_').replace('_a_', '>')}.unique().join(', '),
        formatter.format(map.publicationbegin),
        formatter.format(map.publicationend),
        formatter.format(map.price, 2) ?: '',
        formatter.format(map.promotionalprice, 2) ?: '',
        formatter.format(map.promotionalperiodbegin) ?: '',
        formatter.format(map.promotionalperiodend) ?: '',
        imageNames.join(',')
    ] as String[])
    entityManagerHandler.clear()
} as RowCallbackHandler
jdbc.query(sql, rch)
csv.flush()
zip.closeEntry()

// Write each image
def ai = QAdImage.adImage
entityManagerHandler
        .from(ai)
        .where(ai.ad().publicationPeriod().end.future())
        .iterate(ai)
        .forEachRemaining { AdImage image ->
            def contentType = ContentType.getByMimeType(image.contentType)
            zip.putNextEntry(new ZipEntry("${image.id}.${contentType.extension}"))
            storedFileHandler.getContent(image).withCloseable { content -> IOUtils.copy(content, zip) }
            zip.closeEntry()
            entityManagerHandler.clear()
        }

zip.finish()
zip.close()

return new FileInfo(
        content: new SerializableInputStream(file),
        contentType: ContentType.ZIP.mimeType,
        name: 'ads.zip',
        length: file.length())
Generating an account number for all accounts which doesn’t have a number yet

If the account number is enabled after existing users / transactions, existing accounts will not have numbers automatically assigned. To assign a number to all accounts (even system accounts) which don’t have a number yet, run the following script:

import org.cyclos.entities.banking.QSystemAccount
import org.cyclos.entities.users.QBulkActionUser
import org.cyclos.impl.banking.AccountServiceLocal
import org.cyclos.model.users.bulkactions.AdjustAccountsBulkActionDTO
import org.cyclos.model.users.users.UserQuery

int system = 0
def sa = QSystemAccount.systemAccount
entityManagerHandler
        .from(sa)
        .where(sa.number.isNull())
        .stream(sa)
        .forEach { account ->
            account.number = accountService.generateNumber(account.type, account.owner)
            system++
        }

def query = new UserQuery();
query.setUserStatus(AccountServiceLocal.POSSIBLE_STATUSES_TO_OWN_ACCOUNTS);
def id = bulkActionService.save(new AdjustAccountsBulkActionDTO(query: query));

def bau = QBulkActionUser.bulkActionUser
def users = entityManagerHandler.from(bau).where(bau.bulkAction().id.eq(id)).fetchCount()

return "Generated account numbers for ${system} system accounts" +
        " and scheduled generation for ${users} users"
Generating a custom PDF file

In this example, a custom PDF file is downloaded directly. A similar example could be used as a custom operation:

import org.cyclos.CyclosVersion

import groovy.xml.MarkupBuilder

def out = new StringWriter()

// We'll be using Groovy's MarkupBuilder. Could also be a hand-crafted string
def html = new MarkupBuilder(out)
html.div {
    p "Currently logged-in as ${sessionData.loggedUser?.name}."
    p "This is an example PDF."
    div class:'note', {
        mkp.yield "Built with "
        a href: "https://www.cyclos.org", "Cyclos"
        mkp.yield " version ${CyclosVersion.get()}"
    }
}

def css = """
    .note {
        font-size: 90%;
        color: #333;
        margin-top: 2cm;
        text-align: center;
    }
"""

return pdfHandler
        .newTemplate(out.toString(), css)
        .title("Example PDF")
        // Or, instead of title, hide the header with .noHeader()
        // Similarly, could hide the footer with .noFooter()
        .renderToFile("custom.pdf")
// Alternatively, could use .render() instead of .renderToFile() to get the InputStream
// If the script needs a Base64 version of the content, do:
// Base64.encoder.encodeToString(inputStream.bytes)
Manual account balance verification

Some very large systems may choose to disable the online account balance verification recurring task, by setting cyclos.accountsVerification.balanceCheckDays = never. In such systems it has no effect to manually run the recurring task in the Reports > System information > Recurring tasks tab, because the task is a no-op. Instead, it can be executed by script, as follows:

accountVerificationHandler.fixInconsistentBalances()
Manually rebuilding closed account balances

Account balances are closed daily for accounts that had any transfers in that day. It should never be needed to manually do this operation, but in case of problems, it is possible to completely rebuild the closed balances of accounts since the beginning of history. This operation can take a long time depending on the amount of transfers in the database.

// Rebuild all account balances
accountVerificationHandler.rebuildClosedBalances()

// Rebuild account balances of specific accounts with ids 1 and 2
// accountVerificationHandler.rebuildClosedBalances(1, 2)

4.5. Solutions using scripts

Examples of script types that require a single script can be found directly in the specific script description page (links directly above). Solutions that need several scripts and configurations can be found in this section.

4.5.1. PayPal Integration

It is possible to integrate Cyclos with PayPal, allowing users to top up their account by performing a payment from their PayPal account.

This is done with a custom operation which allows users to confirm the payment in PayPal and then, once the payment is confirmed, a payment from a system account is performed to the corresponding user account, automating the process of buying units. However, keep in mind the rates charged by PayPal, which vary according to some conditions.

To do so, first you’ll need a PayPal premium or business account (for testing – using PayPal sandbox – any account is enough). You’ll need to go to the PayPal Developer page to create an application on "REST API apps", and get the client id and secret.

Then several configurations are required in Cyclos. Scripts can only be created as global administrators logged into a network, so it is advised to use a global admin to perform the configuration. Carefully follow each of the following steps:

Check the root URL

Make sure that the configuration for users use a correct root URL. In System > System configuration > Configurations, select the configuration set for users and make sure the Main URL field points to the correct external URL. It will be used to generate the links which will be sent to PayPal, to redirect users back to Cyclos after confirming / canceling the operation.

Enable transaction number in currency

This can be checked under System > Currencies select the currency used for this operation, mark the Enable transfer number option and fill in the required parameters.

Create a system record type to store the client id and secret

Under System > System configuration > Record types, create a new system record type, with the following characteristics:

  • Name: PayPal Authentication;

  • Internal name: paypalAuth;

  • Display style: Single form;

  • Main menu: System.

For this record type, create the following fields:

  • Client ID:

    • Internal name: clientId;

    • Data type: Single line text;

    • Required: Yes.

  • Client Secret

    • Internal name: clientSecret;

    • Data type: Single line text;

    • Required: Yes.

  • Token:

    • Internal name: token;

    • Data type: Single line text;

    • Required: No.

  • Token expiration:

    • Internal name: tokenExpiration;

    • Data type: Date

    • Required: No.

Create a user record type to store each payment information

Under System > System configuration > Record types, create a new user record type, with the following characteristics:

  • Name: PayPal payment;

  • Internal name: paypalPayment;

  • Display style: List;

  • Main menu: Banking;

  • User management section: Banking.

For this record type, create the following fields:

  • Payment ID:

    • Internal name: paymentId;

    • Data type: Single line text;

    • Required: No.

  • Amount:

    • Internal name: amount;

    • Data type: Decimal;

    • Required: No.

  • Transaction:

    • Internal name: transaction;

    • Data type: Linked entity;

    • Linked entity type: Transaction;

    • Required: No.

Create the library script

Under System > Tools > Scripts, create a new library script, with the following characteristics:

  • Name: PayPal;

  • Type: Library;

  • Included libraries: none;

Script parameters:

# Settings for the access token record type
auth.recordType = paypalAuth
auth.clientId = clientId
auth.clientSecret = clientSecret
auth.token = token
auth.tokenExpiration = tokenExpiration

# Settings for the payment record type
payment.recordType = paypalPayment
payment.paymentId = paymentId
payment.amount = amount
payment.transaction = transaction

# Settings for PayPal
mode = sandbox
currency = EUR
paymentDescription = Buy Cyclos units

# Settings for the Cyclos payment
multiplier = 1
accountType = debitUnits
paymentType = paypalCredits

# Messages
error.invalidRequest = Invalid request
error.transactionNotFound = Transaction not found
error.transactionAlreadyApproved = The transaction was already approved
error.payment = There was an error while processing the payment. Please, try again.
error.notApproved = The payment was not approved
message.canceled = You have cancelled the operation.\nFeel free to start again if needed.
message.done = You have successfully completed the payment. Thank you.

Script code:

import java.nio.charset.StandardCharsets

import org.cyclos.entities.banking.PaymentTransferType
import org.cyclos.entities.banking.SystemAccountType
import org.cyclos.entities.users.RecordCustomField
import org.cyclos.entities.users.SystemRecord
import org.cyclos.entities.users.SystemRecordType
import org.cyclos.entities.users.User
import org.cyclos.entities.users.UserRecord
import org.cyclos.entities.users.UserRecordType
import org.cyclos.impl.banking.PaymentServiceLocal
import org.cyclos.impl.messaging.AlertServiceLocal
import org.cyclos.impl.system.ScriptHelper
import org.cyclos.impl.users.RecordServiceLocal
import org.cyclos.impl.utils.persistence.EntityManagerHandler
import org.cyclos.model.EntityNotFoundException
import org.cyclos.model.banking.accounts.SystemAccountOwner
import org.cyclos.model.banking.transactions.PaymentVO
import org.cyclos.model.banking.transactions.PerformPaymentDTO
import org.cyclos.model.banking.transfertypes.TransferTypeVO
import org.cyclos.model.messaging.alerts.SystemAlertType
import org.cyclos.model.users.records.RecordDataParams
import org.cyclos.model.users.records.UserRecordDTO
import org.cyclos.model.users.recordtypes.RecordTypeVO
import org.cyclos.model.users.users.UserLocatorVO
import org.cyclos.utils.ParameterStorage
import org.springframework.core.ParameterizedTypeReference
import org.springframework.http.HttpEntity
import org.springframework.http.HttpHeaders
import org.springframework.http.HttpMethod
import org.springframework.http.MediaType
import org.springframework.util.LinkedMultiValueMap
import org.springframework.web.client.RestTemplate

import groovy.transform.TypeChecked
import groovy.transform.TypeCheckingMode

/**
 * Class used to store / retrieve the authentication information for PayPal
 * A system record type is used, with the following fields: client id (string),
 * client secret (string), access token (string) and token expiration (date)
 */
@TypeChecked
class PayPalAuth {
    String recordTypeName
    String clientIdName
    String clientSecretName
    String tokenName
    String tokenExpirationName

    SystemRecordType recordType
    SystemRecord record
    Map<String, Object> wrapped

    public PayPalAuth(Map<String, Object> variables) {
        def params = variables.scriptParameters as Map<String, Object>
        recordTypeName = params.'auth.recordType' ?: 'paypalAuth'
        clientIdName = params.'auth.clientId' ?: 'clientId'
        clientSecretName = params.'auth.clientSecret' ?: 'clientSecret'
        tokenName = params.'auth.token' ?: 'token'
        tokenExpirationName = params.'auth.tokenExpiration' ?: 'tokenExpiration'

        // Read the record type and the parameters for field internal names
        recordType = (variables.entityManagerHandler as EntityManagerHandler)
                .find(SystemRecordType, recordTypeName)

        // Should return the existing instance, of a single form type.
        // Otherwise it would be an error
        def dataParams =
                new RecordDataParams(recordType: new RecordTypeVO(id: recordType.id))
        record = (variables.recordService as RecordServiceLocal)
                .newEntity(dataParams) as SystemRecord
        if (!record.persistent) throw new IllegalStateException(
            "No instance of system record ${recordType.name} was found")

        wrapped = (variables.scriptHelper as ScriptHelper).wrap(record, recordType.fields)
    }

    public String getClientId() {
        wrapped[clientIdName]
    }
    public String getClientSecret() {
        wrapped[clientSecretName]
    }
    public String getToken() {
        wrapped[tokenName]
    }
    public Date getTokenExpiration() {
        wrapped[tokenExpirationName] as Date
    }
    public void setClientId(String clientId) {
        wrapped[clientIdName] = clientId
    }
    public void setClientSecret(String clientSecret) {
        wrapped[clientSecretName] = clientSecret
    }
    public void setToken(String token) {
        wrapped[tokenName] = token
    }
    public void setTokenExpiration(Date tokenExpiration) {
        wrapped[tokenExpirationName] = tokenExpiration
    }
}

/**
 * Class used to store / retrieve PayPal payments as user records in Cyclos
 */
@TypeChecked
class PayPalRecord {
    String recordTypeName
    String paymentIdName
    String amountName
    String transactionName

    UserRecordType recordType
    Map<String, RecordCustomField> fields

    private EntityManagerHandler entityManagerHandler
    private RecordServiceLocal recordService
    private ScriptHelper scriptHelper

    public PayPalRecord(Map<String, Object> variables) {
        def params = variables.scriptParameters as Map<String, Object>
        recordTypeName = params.'payment.recordType' ?: 'paypalPayment'
        paymentIdName = params.'payment.paymentId' ?: 'paymentId'
        amountName = params.'payment.amount' ?: 'amount'
        transactionName = params.'payment.transaction' ?: 'transaction'

        entityManagerHandler = variables.entityManagerHandler as EntityManagerHandler
        recordService = variables.recordService as RecordServiceLocal
        scriptHelper = variables.scriptHelper as ScriptHelper
        recordType = entityManagerHandler.find(UserRecordType, recordTypeName)
        fields = [:]
        recordType.fields.each {f -> fields[f.internalName] = f}
    }

    /**
     * Creates a payment record, for the given user and JSON,
     * as returned from PayPal's create payment REST method
     */
    public UserRecord create(User user, Number amount) {
        RecordDataParams newParams = new RecordDataParams([
            user: new UserLocatorVO(user.id),
            recordType: new RecordTypeVO(recordType.id)])
        def dto = recordService.getDataForNew(newParams).dto as UserRecordDTO
        def wrapped = scriptHelper.wrap(dto, recordType.fields)
        wrapped[amountName] = amount
        UserRecord record = recordService.saveEntity(dto)
        return record
    }

    /**
     * Finds the record by id
     */
    public UserRecord find(Long id) {
        try {
            UserRecord userRecord = entityManagerHandler.find(UserRecord, id)
            if (userRecord.type != recordType) {
                return null
            }
            return userRecord
        } catch (EntityNotFoundException e) {
            return null
        }
    }

    /**
     * Removes the given record, but only if it is of the
     * expected type and hasn't been confirmed
     */
    public void remove(UserRecord userRecord) {
        if (userRecord.type != recordType) {
            return
        }
        Map<String, Object> wrapped = scriptHelper
                .wrap(userRecord, recordType.fields)
        if (wrapped[transactionName] != null) return
            entityManagerHandler.remove(userRecord)
    }
}

/**
 * Class used to interact with PayPal services
 */
@TypeChecked
class PayPalService {
    String mode
    String baseUrl
    String currency
    String paymentDescription

    String accountTypeName
    String paymentTypeName
    double multiplier

    SystemAccountType accountType
    PaymentTransferType paymentType
    PayPalAuth auth
    PayPalRecord record

    private ScriptHelper scriptHelper
    private PaymentServiceLocal paymentService
    private AlertServiceLocal alertService
    private ParameterStorage storage
    private Map<String, Object> params
    private RestTemplate rest

    PayPalService(Map<String, Object> variables) {
        this.auth = new PayPalAuth(variables)
        try {
            this.record = new PayPalRecord(variables)
        } catch(EntityNotFoundException ex) {
            // There are usages without user record
        }
        scriptHelper = variables.scriptHelper as ScriptHelper
        paymentService = variables.paymentService as PaymentServiceLocal
        alertService = variables.alertService as AlertServiceLocal
        storage = variables.parameterStorage as ParameterStorage
        params = variables.scriptParameters as Map<String, Object>
        rest = variables.rest as RestTemplate

        mode = params.mode ?: 'sandbox'
        if (mode != 'sandbox' && mode != 'live') {
            throw new IllegalArgumentException("Invalid PayPal parameter " +
            "'mode': ${mode}. Should be either sandbox or live")
        }
        baseUrl = mode == 'sandbox'
                ? 'https://api.sandbox.paypal.com' : 'https://api.paypal.com'

        currency = params.currency
        if (currency == null || currency.empty) {
            throw new IllegalArgumentException("Missing PayPal parameter 'currency'")
        }

        def emh = variables.entityManagerHandler as EntityManagerHandler
        accountTypeName = params.accountType
        if (accountTypeName == null || accountTypeName.empty)
            throw new IllegalArgumentException("Missing PayPal parameter 'accountType'")
        paymentTypeName = params.paymentType
        if (paymentTypeName == null || paymentTypeName.empty)
            throw new IllegalArgumentException("Missing PayPal parameter 'paymentType'")
        accountType = emh.find(SystemAccountType, accountTypeName)
        if (!accountType.currency.transactionNumber?.used) {
            throw new IllegalStateException("Currency " + accountType.currency
            + " doesn't have transaction number enabled")
        }
        paymentType = emh.find(PaymentTransferType, paymentTypeName, accountType)
        multiplier = Double.parseDouble((params.multiplier as String) ?: "1")
        paymentDescription = params.paymentDescription ?: ""
    }

    /**
     * Creates an order in PayPal and the corresponding user record
     */
    Map<String, Object> createOrder(User user, Number amount, String callbackUrl) {
        // Create the UserRecord for this payment
        UserRecord userRecord = record.create(user, amount)
        //store the record's id to retrieve it after the payment was confirmed in PayPal
        storage['recordId'] = userRecord.id

        // Create the payment in PayPal
        def json = createOrder(amount, callbackUrl)
        //store the PayPal order id to retrieve it after the payment was confirmed in PayPal
        storage['orderId'] = json.id

        return json
    }

    /**
     * Creates an order in PayPal with a given amount, without updating any record
     */
    Map<String, Object> createOrder(Number amount, String callbackUrl) {
        callbackUrl += callbackUrl.contains("?") ? "&" : "?"
        String returnUrl = "${callbackUrl}success=true"
        String cancelUrl = "${callbackUrl}cancel=true"

        def jsonBody = [
            intent: "CAPTURE",
            application_context: [
                return_url: returnUrl,
                cancel_url: cancelUrl,
                user_action: "PAY_NOW"
            ],
            purchase_units: [
                [
                    description: paymentDescription,
                    amount: [
                        value: amount,
                        currency_code: currency
                    ]
                ]
            ]
        ]
        // Create the payment in PayPal
        return performRequest("${baseUrl}/v2/checkout/orders", jsonBody, HttpMethod.POST)
    }

    /**
     * Capture the order (execute the payment in PayPal)
     */
    Map<String, Object> captureOrder(String orderId) {
        return performRequest("${baseUrl}/v2/checkout/orders/${orderId}/capture", null, HttpMethod.POST)
    }

    /**
     * Get the order information from PayPal
     */
    Map<String, Object> getOrderFromPayPal(String orderId) {
        return performRequest("${baseUrl}/v2/checkout/orders/${orderId}", null, HttpMethod.GET)
    }

    /**
     * Executes a PayPal payment, and creates the payment in Cyclos
     */
    Map<String, Object> execute(UserRecord userRecord) {
        def wrapped = scriptHelper.wrap(userRecord)
        def orderId = storage['orderId'] as String
        // Execute the payment in PayPal
        def capturedOrder = captureOrder(orderId) as Map<String, Object>
        // Update the payment id
        wrapped[record.paymentIdName] = getPaymentIdFromCapturedOrder(capturedOrder)
        def vo
        try {
            // Try to perform the payment in Cyclos, if it fails, refund the payment in PayPal
            vo = perform(capturedOrder, userRecord.user)
        } catch (Exception ex) {
            refundCapturedOrder(capturedOrder, userRecord, userRecord.user)
            throw ex
        }
        if (vo != null) {
            // Update the record, setting the linked transaction
            wrapped[record.transactionName] = vo
            userRecord.lastModifiedDate = new Date()
        }
        return capturedOrder
    }
    /**
     * Refund the completed order using the refund link returned when the
     * order was captured and remove the user record if given
     */
    void refundCapturedOrder(Map<String, Object> capturedOrder, UserRecord userRecord, User user) {
        def refundLink = getRefundLinkFromCapturedOrder(capturedOrder)
        if (refundLink) {
            def refundedOrder
            try {
                // Make the refund
                refundedOrder = performRequest(refundLink, null, HttpMethod.POST)
            } catch (Exception ex) {
                //Do nothing because an alert is going to be created
            }

            if (!refundedOrder || refundedOrder.status != "COMPLETED") {
                createRefundFailAlert(capturedOrder, user)
            } else if (userRecord) {
                record.remove(userRecord)
            }
        }
    }
    /**
     * Create the system alert for a failed refund
     */
    private void createRefundFailAlert(Map<String, Object> capturedOrder, User user) {
        def errorMessage = """User: ${user.username}. The PayPal payment
            (${getPaymentIdFromCapturedOrder(capturedOrder)}) was completed,
            but there was an error in Cyclos and the attempt to refund in PayPal failed."""
        alertService.create(SystemAlertType.CUSTOM, errorMessage)
    }

    /**
     * Performs the payment in Cyclos
     */
    PaymentVO perform(Map<String, Object> capturedOrder, User subject) {
        if (getPaymentStatusFromCapturedOrder(capturedOrder) == 'COMPLETED') {
            def amount = new BigDecimal(getAmountFromCapturedOrder(capturedOrder) as String)
            BigDecimal finalAmount = amount * multiplier
            // Perform the payment in Cyclos
            PerformPaymentDTO dto = new PerformPaymentDTO()
            dto.owner = SystemAccountOwner.instance()
            dto.subject = subject
            dto.amount = finalAmount
            dto.type = new TransferTypeVO(paymentType.id)
            return paymentService.perform(dto)
        } else {
            return null
        }
    }

    /**
     * Get the payment id of a captured order result
     */
    @TypeChecked(TypeCheckingMode.SKIP)
    String getPaymentIdFromCapturedOrder(capturedOrder) {
        return capturedOrder.purchase_units[0].payments.captures[0].id
    }

    /**
     * Get the payment status of a captured order result.
     * We must get the status from the captured payment
     * because it will be NOT completed even when the order status is completed
     */
    @TypeChecked(TypeCheckingMode.SKIP)
    String getPaymentStatusFromCapturedOrder(capturedOrder) {
        return capturedOrder.purchase_units[0].payments.captures[0].status
    }

    /**
     * Get the amount of a captured order result
     */
    @TypeChecked(TypeCheckingMode.SKIP)
    String getAmountFromCapturedOrder(capturedOrder) {
        return capturedOrder.purchase_units[0].payments.captures[0].amount.value
    }

    /**
     * Get the refund link of a captured order result
     */
    @TypeChecked(TypeCheckingMode.SKIP)
    String getRefundLinkFromCapturedOrder(capturedOrder) {
        return capturedOrder.purchase_units[0].payments.captures[0].links.find {it.rel == "refund"}?.href
    }

    /**
     * Performs a synchronous request, posting and accepting JSON
     */
    Map<String, Object> performRequest(String url, def jsonBody, HttpMethod method) {
        // Check if we need a new token
        if (auth.token == null || auth.tokenExpiration < new Date()) {
            refreshToken()
        }

        // Send the request
        def headers = new HttpHeaders()
        headers.setContentType(MediaType.APPLICATION_JSON)
        headers.setBearerAuth(auth.token)
        def responseType = new ParameterizedTypeReference<Map>() {}
        return rest.exchange(url, method, new HttpEntity(jsonBody, headers), responseType).body;
    }

    /**
     * Refreshes the access token
     */
    private void refreshToken() {
        def url = "${baseUrl}/v1/oauth2/token"
        def headers = new HttpHeaders()
        headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED)
        headers.setBasicAuth(auth.clientId, auth.clientSecret, StandardCharsets.UTF_8)
        def body = new LinkedMultiValueMap<String, String>()
        body.put("grant_type", ["client_credentials"])
        def response = rest.postForObject(url, new HttpEntity(body, headers), Map) as Map<String, Object>

        // Update the authentication data
        auth.token = response.access_token
        auth.tokenExpiration = new Date(System.currentTimeMillis() +
                (((response.expires_in as Integer) - 30) * 1000))
    }
}
Create the custom operation script

Under System > Tools > Scripts, create a new custom operation script, with the following characteristics:

  • Name: Buy units with PayPal;

  • Type: Custom operation;

  • Run with all permissions: Yes;

  • Included libraries: PayPal;

  • Parameters: leave empty.

Script code executed when the custom operation is executed:

import org.cyclos.entities.users.User

import groovy.transform.TypeChecked

@TypeChecked
def createPayment(){
    def variables = binding.variables
    def formParameters = variables.formParameters as Map<String, Object>
    def service = new PayPalService(variables)

    def user = variables.user as User
    def amount = formParameters.amount as Number
    def returnUrl = variables.returnUrl as String
    def result = service.createOrder(user, amount, returnUrl)

    def links = result.links as Map<String, Object>[]
    def link = links.find {it.rel == "approve"}
    if (link) {
        return link.href
    } else {
        throw new IllegalStateException("No approval url returned from PayPal")
    }
}

createPayment()

Script code executed when the external site redirects the user back to Cyclos:

import org.cyclos.entities.users.UserRecord
import org.cyclos.impl.system.ScriptHelper
import org.cyclos.model.utils.RequestInfo
import org.cyclos.server.utils.ObjectParameterStorage

import groovy.transform.TypeChecked

@TypeChecked
def payPalCallback() {
    def variables = binding.variables as Map<String, Object>
    def scriptParameters = variables.scriptParameters as Map<String, Object>
    def service = new PayPalService(variables)
    def storage = variables.storage as ObjectParameterStorage
    def recordId = storage['recordId'] as Long
    def request = variables.request as RequestInfo
    def record = service.record

    // No record?
    if (recordId == null) {
        return "[ERROR] " +
                (scriptParameters.'error.invalidRequest' ?: "Invalid request")
    }

    // Find the corresponding record
    UserRecord userRecord = record.find(recordId)
    if (userRecord == null) {
        return "[ERROR] " +
                (scriptParameters.'error.transactionNotFound'
                ?: "Transaction not found")
    }
    def wrapped = (variables.scriptHelper as ScriptHelper).wrap(userRecord)

    if (request.getParameter("cancel")) {
        // The operation has been canceled.
        // Remove the record and send a message.
        record.remove(userRecord)
        return "[WARN]" + scriptParameters.'message.canceled'
                ?: "You have cancelled the operation.\nFeel free to start again if needed."
    } else {
        // Execute the payment
        try {
            def order = service.execute(userRecord)
            if (service.getPaymentStatusFromCapturedOrder(order) == 'COMPLETED') {
                return scriptParameters.'message.done'
                        ?: "You have successfully completed the payment. Thank you."
            } else {
                return "[ERROR] " + scriptParameters.'error.notApproved'
                        ?: "The payment was not approved"
            }
        } catch (Exception e) {
            return "[ERROR] " + scriptParameters.'error.payment'
                    ?: "There was an error while processing the payment. Please, try again."
        }
    }
}

payPalCallback()
Create the custom operation

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Buy units with PayPal (can be changed – will be the label displayed on the menu);

  • Enabled: Yes;

  • Scope: User;

  • Script: Buy units with PayPal;

  • Script parameters: leave empty;

  • Result type: External redirect;

  • Has file upload: no;

  • Main menu: Banking;

  • User management section: Banking;

  • Information text: You can add here some text explaining the process – it will be displayed in the operation page;

  • Confirmation text: Leave empty (can be used to show a dialog asking the user to confirm before submitting, but in this case is not needed).

For this custom operation, create the following form field:

  • Name: Amount;

  • Internal name: amount;

  • Data type: Decimal;

  • Decimal digits: 2;

  • Required: yes.

Configure the system account from which payments will be performed to users

Under System > Accounts configuration > Account types, choose the (normally unlimited) account from which payments will be performed to users. Then set its internal name to some meaningful name. The example configuration uses debitUnits as internal name, but it can be changed. Save the form.

Configure the payment type which will be used on payments

Still in the details page for the account type, on the Transfer types tab, create a new Payment transfer type with the following characteristics:

  • Name: Units bought with PayPal (can be changed as desired);

  • Internal name: paypalCredits;

  • To: Select the user account which will receive the payment;

  • Enabled: Yes.

Grant the administrator permissions

Under System > User configuration > Groups, select the Network administrators group. Then, in the Permissions tab:

  • In System > System records, set the permissions to view, create and edit for the PayPal authentication record;

  • In User data > User records, make the PayPal payment visible only (make sure to create, edit and remove are unchecked, as this record is not meant to be manually edited);

  • Save the permissions.

Setup the PayPal credentials

Click System > System records > PayPal authentication. If this menu entry is not showing up, refresh the browser page (by pressing F5) and try again. Update the Client ID and Client Secret fields exactly with the ones you got in the application you registered in the PayPal Developer page.

Remember that PayPal has a sandbox, which can be used to test the application, and a live environment. For now, use the sandbox credentials. The other 2 fields can be left blank. Save the record.

Once the record is properly set, if you want to remove it from the menu, you can just remove the permission to view this system record in the administrator group page.

Grant the user permissions / enable the operation

In System > User configuration > Products (permissions), select the member product for users which will run the operation.

  • In the Custom operations field, make the Buy units with PayPal both enabled and allowed to run;

  • In Records, enable the PayPal payment record. It can be made visible to the users themselves. If not, only admins will be able to see the records;

  • Save the product. From this moment, the operation will show up for users in the banking menu.

Configuring the script parameters

In the PayPal library script, in parameters, there are several configurations which can be done. All those settings can be overridden in the custom operation’s script parameters, allowing using distinct configurations for distinct operations.

For example, it is possible to have distinct operations to perform payments in distinct currencies. In that case, the script parameters for each operation would define the currency again.

Here are some elements which can be configured:

  • Internal names for the records used to store the credentials and payments.

  • PayPal mode: the mode parameter can be either sandbox or live, indicating that operations are performed either in a test or in the real environment. To go live, you’ll need a premier or business account in PayPal, and you need to use the live credentials (client ID and client secret) in Cyclos.

  • Payment currency: the currency parameter defines the 3-letter ISO 4217 code for the currency in PayPal. Sometimes, according to country-specific laws, the currency used for payments may be limited. For example, Brazilians can only pay other Brazilians in Reais. Make sure the PayPal destination account can receive payments for the specified currency, otherwise payments or refunds will fail;

  • Description for payments in PayPal: using the paymentDescription setting.

  • Amount multiplier: Sometimes it may be desired that the payment performed in Cyclos isn’t of the exact amount of the payment in PayPal. This can normally be resolved using transfer fees, but it could also be handy to use this parameter called multiplier. If left in 1, the payment in Cyclos will have the same amount as the one in PayPal. If greater / less than 1, the payment in Cyclos will be greater / less than the one in PayPal. For example, if the multiplier is 1.05, and the PayPal payment was 100 USD, the payment in Cyclos will have the amount 105. Or, if the multiplier is 0.95 and the PayPal payment was 200 EUR, the payment in Cyclos will be of 190.

  • System account from which the payment will be performed to users: the accountType setting is the internal name of the system account type from which payments will be performed, as explained previously. Make sure it is exactly the same as set in the account type.

  • Payment type: the paymentType setting is the internal name of the payment transfer type used. Make sure it is exactly the same internal name set in the payment type that was created in previous steps.

  • Messages: several messages (displayed to the user) can be set / translated here.

Other considerations

Make sure the payment type is from an unlimited account, so payments in Cyclos won’t fail because of funds. The way the example script is done, first the payment is executed in PayPal and, if authorized, a payment is made in Cyclos. If this payment fails, to avoid inconsistency between the Cyclos account and the PayPal payment, a refund payment is performed in PayPal. If that refund fails, it creates a Custom system alert, so it is advisable to have admins receive that type of alert.

4.5.2. Loan module

Loan features in Cyclos 4 can be implemented using scripting. As loans tend to be very specific for each project, having it implemented with scripts brings the possibility to tailor the behavior to each project.

The example provided works as follows:

  • An administrator has a custom operation to grant the loan, setting the amount, number of installments and first installment date.

  • The loan is a payment from a system account to a user. It has a status, which can be either open or closed.

  • The same custom operation also performs a scheduled payment from the user to the system, with each installment amount and due date corresponding to the loan installments. This scheduled payment has (with a custom field) a link to the original loan. Also, the loan payment has a link to the scheduled payment, making it easy to navigate between them. However, if the loan is pending authorization, the scheduled payment won’t be created.

  • Each installment will be processed at the respective due date, allowing users to repay the loan with internal units. The administrator can, however, mark individual installments as settled, which means the installment won’t be repaid internally, but with some other way (for example, with money or using other Cyclos payments).

  • Once the scheduled payment is closed, an extension point updates the status of the original payment to closed.

  • If the original loan was submitted for authorization, an extension point is triggered when it is authorized, and then creates the scheduled payment. IMPORTANT: If the administrator performing a payment also has the permission to authorize it, the payment will be immediately processed. So, be careful when testing with a single administrator group when authorization is desired, as in that case the loan would never get to authorization.

In order to configure the loan script, follow carefully each of the following steps:

Enable transaction number in currency

This can be checked under System > Currencies select the currency used for this operation, mark the Enable transfer number option and fill in the required parameters.

Create the transfer status flow

Under System > Accounts configuration > Transfer status flows, create a new one, with the following characteristics:

  • Name: Loan status (can be changed as desired);

  • Internal name: loan (this name is used in the example configuration).

After saving, create the following statuses:

  • Closed:

    • Internal name: closed;

    • Possible next statuses: <None>.

  • Open:

    • Internal name: open;

    • Possible next statuses: Closed.

Create the payment custom fields

Under System > Accounts configuration > Payment fields, create a new one, with the following fields:

  • Installments count:

    • Internal name: numberOfInstallments;

    • Data type: Integer;

    • Required: Yes.

  • First due date:

    • Internal name: firstDueDate;

    • Data type: Date;

    • Required: Yes.

  • Loan:

    • Internal name: loan;

    • Data type: Linked entity;

    • Linked entity type: Transaction;

    • Required: No.

  • Repayment:

    • Internal name: repayment;

    • Data type: Linked entity;

    • Linked entity type: Transaction;

    • Required: No.

Configure the system account from which payments will be performed to users

Under System > Accounts configuration > Account types, choose the (normally unlimited) account from which payments will be performed to users. Then set its internal name to some meaningful name. The example configuration uses debitUnits as internal name, but it can be changed later. Save the form.

Create the payment type which will be used to grant the loan

Still in the system account type details page for the account type, on the Transfer types tab, create a new Payment transfer type with the following characteristics:

  • Name: Loan (can be changed as desired);

  • Internal name: loanGrant;

  • Default description: Loan grant (can be changed as desired, is the description for payments, visible in the account history);

  • To: select the user account which will receive the payment;

  • Transfer status flows: Loan status;

  • Initial status for Loan status: Open;

  • Enabled: Yes.

After saving, on the "Payment fields" tab, add the following custom fields:

  • Installments count;

  • First due date;

  • Repayment.

If the loan can go through authorization, then create an authorization role in System > Account configuration > Authorization roles. Then, in the payment type details, check the "Requires authorization" field. After saving, in the "Authorization levels" tab, add a new authorization level with that role. Afterwards, grant some administrator group the permission to manage that authorization role.

Configure the user account which will receive loans

Under System > Accounts configuration > Account types, choose the user account which will receive payments. Then set its internal name to some meaningful name. The example configuration uses userUnits as internal name. Save the form.

Create the payment type which will be used to repay the loan

Still in the user account type details page, on the Transfer types tab, create a new Payment transfer type with the following characteristics:

  • Name: Loan repayment (can be changed as desired);

  • Internal name: loanRepayment;

  • Default description: Loan repayment (can be changed as desired, is the description for payments, visible in the account history);

  • To: select the system account which granted the loan;

  • Enabled: Yes;

  • Allows scheduled payment: Yes;

  • Max installments on scheduled payments: 36 (any value greater than zero is fine);

  • Show scheduled payments to receiver: Yes;

  • Reserve total amount on scheduled payments: No.

After saving, on the Payment fields tab, add the custom field named "Loan".

Create the library script

Under System > Tools > Scripts, create a new library script, with the following script parameters:

# Loan configuration
loan.account = debitUnits
loan.type = loanGrant
#loan.description =

# Repayment configuration
repayment.account = userUnits
repayment.type = loanRepayment
#repayment.description =

# Payment custom fields
field.loan = loan
field.repayment = repayment

# Monthly compound interest rate (zero for none)
monthlyInterestRate = 0

# Transfer status configuration
status.flow = loan
status.open = open
status.closed = closed

# Custom operation configuration
operation.amount = amount
operation.installments = numberOfInstallments
operation.firstDueDate = firstDueDate

# Messages
message.invalidInstallments = The number of installments is invalid
message.invalidLoanAmount = Invalid loan amount
message.invalidFirstDueDate = The first due date cannot be lower than tomorrow
message.loanGranted = The loan was successfully granted
message.loanGranted.pending = The loan was granted and is now pending authorization
message.authorization.expired = The loan cannot be authorized as the first due date is over

The script code is:

import org.cyclos.entities.banking.Payment
import org.cyclos.entities.banking.PaymentTransferType
import org.cyclos.entities.banking.ScheduledPayment
import org.cyclos.entities.banking.SystemAccountType
import org.cyclos.entities.banking.TransactionCustomField
import org.cyclos.entities.banking.Transfer
import org.cyclos.entities.banking.TransferStatus
import org.cyclos.entities.banking.TransferStatusFlow
import org.cyclos.entities.banking.UserAccountType
import org.cyclos.entities.users.User
import org.cyclos.impl.access.SessionData
import org.cyclos.impl.banking.PaymentServiceLocal
import org.cyclos.impl.banking.ScheduledPaymentServiceLocal
import org.cyclos.impl.banking.TransferStatusServiceLocal
import org.cyclos.impl.system.ConfigurationAccessor
import org.cyclos.impl.system.ScriptHelper
import org.cyclos.impl.utils.persistence.EntityManagerHandler
import org.cyclos.model.ValidationException
import org.cyclos.model.banking.accounts.SystemAccountOwner
import org.cyclos.model.banking.transactions.InstallmentDTO
import org.cyclos.model.banking.transactions.PaymentVO
import org.cyclos.model.banking.transactions.PerformPaymentDTO
import org.cyclos.model.banking.transactions.PerformScheduledPaymentDTO
import org.cyclos.model.banking.transactions.ScheduledPaymentVO
import org.cyclos.model.banking.transfers.TransferVO
import org.cyclos.model.banking.transferstatus.ChangeTransferStatusDTO
import org.cyclos.model.banking.transferstatus.TransferStatusVO
import org.cyclos.model.banking.transfertypes.TransferTypeVO
import org.cyclos.model.utils.TimeField
import org.cyclos.server.utils.DateHelper
import org.cyclos.utils.BigDecimalHelper

import groovy.transform.TypeChecked



@TypeChecked
class Loan {
    Map<String, Object> config
    EntityManagerHandler emh
    PaymentServiceLocal paymentService
    ScheduledPaymentServiceLocal scheduledPaymentService
    TransferStatusServiceLocal transferStatusService
    ScriptHelper scriptHelper
    ConfigurationAccessor configuration

    double monthlyInterestRate
    SystemAccountType systemAccount
    UserAccountType userAccount
    PaymentTransferType loanType
    PaymentTransferType repaymentType
    TransactionCustomField loanField
    TransactionCustomField repaymentField
    TransferStatusFlow flow
    TransferStatus open
    TransferStatus closed

    Loan(Binding binding) {
        def variables = binding.variables
        config = [:]
        def params = variables.scriptParameters as Map<String, Object>
        [
            'loan.account': 'systemAccount',
            'loan.type': 'loanGrant',
            'loan.description': null,
            'repayment.account': 'userUnits',
            'repayment.type': 'loanRepayment',
            'repayment.description': null,
            'field.loan': 'loan',
            'field.repayment': 'repayment',
            'monthlyInterestRate' : null,
            'status.flow': 'loan',
            'status.open': 'open',
            'status.closed': 'closed',
            'operation.amount': 'amount',
            'operation.installments': 'installments',
            'operation.firstDueDate': 'firstDueDate',
            'message.invalidInstallments':
            'The number of installments is invalid',
            'message.invalidLoanAmount': 'Invalid loan amount',
            'message.invalidFirstDueDate':
            'The first due date cannot be lower than tomorrow',
            'message.loanGranted':
            'The loan was successfully granted to the user',
            'message.loanGranted.pending':
            'The loan was granted and is now pending authorization',
            'message.authorization.expired':
            'The loan cannot be authorized as the first due date is over'
        ].each { k, v ->
            config[k] = params[k] ?: v
        }
        emh = variables.entityManagerHandler as EntityManagerHandler
        paymentService = variables.paymentService as PaymentServiceLocal
        scriptHelper = variables.scriptHelper as ScriptHelper
        scheduledPaymentService =
                variables.scheduledPaymentService as ScheduledPaymentServiceLocal
        transferStatusService =
                variables.transferStatusService as TransferStatusServiceLocal
        configuration =
                (variables.sessionData as SessionData).configuration as ConfigurationAccessor

        systemAccount = emh.find(SystemAccountType, config.'loan.account' as String)
        if (systemAccount.currency.transactionNumber == null
                || !systemAccount.currency.transactionNumber.used) {
            throw new IllegalStateException("The currency ${systemAccount.currency.name}"
            + " doesn't have transaction number enabled")
        }
        userAccount = emh.find(UserAccountType, config.'repayment.account' as String)
        loanType =
                emh.find(PaymentTransferType, config.'loan.type' as String, systemAccount)
        repaymentType =
                emh.find(PaymentTransferType, config.'repayment.type' as String, userAccount)
        if (!repaymentType.allowsScheduledPayments) {
            throw new IllegalStateException(
            "The repayment type ${repaymentType.name} doesn't allows scheduled payment")
        }

        loanField = emh.find(TransactionCustomField, config.'field.loan' as String)
        repaymentField =
                emh.find(TransactionCustomField, config.'field.repayment' as String)
        if (!loanType.customFields.contains(repaymentField)) {
            throw new IllegalStateException("The loan type ${loanType.name}"
            + " doesn't contain the custom field ${repaymentField.name}")
        }
        if (!repaymentType.customFields.contains(loanField)) {
            throw new IllegalStateException("The repayment type ${repaymentType.name}"
            + " doesn't contain the custom field ${loanField.name}")
        }
        flow = emh.find(TransferStatusFlow, config.'status.flow' as String)
        open = emh.find(TransferStatus,config.'status.open' as String, flow)
        closed = emh.find(TransferStatus, config.'status.closed' as String, flow)
        monthlyInterestRate = (config.'monthlyInterestRate' as String)?.toDouble() ?: 0
    }

    BigDecimal calculateInstallmentAmount(BigDecimal amount, int installments,
            Date grantDate, Date firstInstallmentDate) {

        // Calculate the delay
        Date shouldBeFirstExpiration = DateHelper.add(grantDate, TimeField.DAYS, 30)
        int delay = (int) DateHelper.daysBetween(firstInstallmentDate, shouldBeFirstExpiration)
        if (delay < 0) {
            delay = 0
        }
        double interest = monthlyInterestRate / 100.0
        double numerator = ((1 + interest) ** (installments + delay / 30.0)) * interest
        double denominator = ((1 + interest) ** installments) - 1
        BigDecimal result = amount * numerator / denominator
        return BigDecimalHelper.round(result, systemAccount.currency.precision)
    }

    void close(ScheduledPayment scheduledPayment) {
        def map = scriptHelper.wrap(scheduledPayment)
        Payment loan = map.get(loanField.internalName) as Payment
        Transfer loanTransfer = loan.transfer
        TransferStatus status = loanTransfer.getStatus(flow)
        if (status != closed) {
            // The loan was not closed: close it
            transferStatusService.changeStatus(new ChangeTransferStatusDTO([
                transfer: new TransferVO(loanTransfer.id),
                newStatus: new TransferStatusVO(closed.id)
            ]))
        }
    }

    Payment grant(User user, Map<String, Object> formParameters) {
        BigDecimal loanAmount = formParameters[config.'operation.amount'] as BigDecimal
        int installments = formParameters[config.'operation.installments'] as int
        Date firstDueDate = formParameters[config.'operation.firstDueDate'] as Date
        Date minDate = DateHelper.shiftToNextDay(new Date(), configuration.timeZone)
        if (installments < 1 || installments > repaymentType.maxInstallments)
            throw new ValidationException(config.'message.invalidInstallments' as String)
        if (loanAmount < 1)
            throw new ValidationException(config.'message.invalidLoanAmount' as String)
        if (firstDueDate < minDate)
            throw new ValidationException(config.'message.invalidFirstDueDate' as String)

        // Grant the loan, copying the installments count and first due date
        PerformPaymentDTO perform = new PerformPaymentDTO([
            owner: SystemAccountOwner.instance(),
            subject: user,
            type: new TransferTypeVO(loanType.id),
            amount: loanAmount,
            description: config.'loan.description' as String
        ])
        def performBean = scriptHelper.wrap(perform)
        performBean[config.'operation.installments' as String] = installments
        performBean[config.'operation.firstDueDate' as String] = firstDueDate
        PaymentVO loanVO = paymentService.perform(perform)
        Payment loan = emh.find(Payment, loanVO.id)
        if (loan.transfer != null) {
            // The loan is processed. Create the repayment
            createRepayment(loan)
        }
        return loan
    }

    ScheduledPayment createRepayment(Payment payment) {
        Transfer loanTransfer = payment.transfer
        if (loanTransfer == null) {
            return null
        }
        TransferStatus currentStatus = loanTransfer.getStatus(flow)
        if (currentStatus != open) {
            throw new ValidationException(
            "The initial status for flow ${flow.name} in ${loanType.name} "
            + "is not the expected one: ${open.name}, but ${currentStatus?.name} instead")
        }

        // Read the scheduling information from the loan
        def loanBean = scriptHelper.wrap(payment)
        def existingRepayment = loanBean[repaymentField.internalName]
        if (existingRepayment != null) {
            return existingRepayment as ScheduledPayment
        }
        BigDecimal loanAmount = payment.amount
        Integer installments = loanBean[config.'operation.installments'] as Integer
        Date firstDueDate = loanBean[config.'operation.firstDueDate'] as Date

        // Make sure the first due date is not expired
        Date now = new Date()
        if (firstDueDate.before(now)) {
            throw new ValidationException(config.'message.authorization.expired' as String)
        }

        // Perform the repayment scheduled payment
        PerformScheduledPaymentDTO dto = new PerformScheduledPaymentDTO([
            from: payment.toOwner,
            to: payment.fromOwner,
            type: new TransferTypeVO(repaymentType.id),
            amount: payment.amount,
            description: config.'repayment.description' as String
        ])
        def dtoBean = scriptHelper.wrap(dto)
        dtoBean.installmentsCount = installments
        dtoBean.firstInstallmentDate = firstDueDate
        dtoBean[loanField.internalName] = payment

        // Interest
        if (monthlyInterestRate > 0.00001) {
            BigDecimal installmentAmount = calculateInstallmentAmount(
                    loanAmount, installments, new Date(), firstDueDate)

            dto.installments = []
            Date dueDate = firstDueDate
            for (int i = 0; i < installments; i++) {
                def installment = new InstallmentDTO()
                def instBean = scriptHelper.wrap(installment)
                instBean.dueDate = dueDate
                instBean.amount = installmentAmount
                dto.installments << installment
                dueDate = DateHelper.add(dueDate, TimeField.DAYS, 30)
            }
            dtoBean.amount = installmentAmount * installments
        }

        ScheduledPaymentVO repaymentVO = scheduledPaymentService.perform(dto)
        ScheduledPayment repayment = emh.find(ScheduledPayment, repaymentVO.id)

        // Update the loan with the repayment link
        loanBean[repaymentField.internalName] = repayment
        return repayment
    }
}

binding.variables.loan = new Loan(binding)
Create the custom operation script

Create a new script for the custom operation, with the following characteristics:

  • Name: Grant loan;

  • Type: Custom operation;

  • Included libraries: Loan;

  • Run with all permissions: No;

  • Parameters: leave empty.

Script code executed when the custom operation is executed:

import org.cyclos.entities.banking.Payment
import org.cyclos.entities.users.User

import groovy.transform.TypeChecked

@TypeChecked
def grantLoan() {
    def variables = binding.variables
    Loan loan = variables.loan as Loan
    Payment payment = loan.grant(variables.user as User,
        variables.formParameters as Map<String, Object>)
    if (payment.transfer == null) {
        return loan.config['message.loanGranted.pending']
    } else {
        return loan.config['message.loanGranted']
    }
}

grantLoan()
Create two extension point scripts

Create a new script for the transaction extension point, which will close the loan when all installments are processed:

  • Name: Loan closing;

  • Type: Extension point;

  • Included libraries: Loan;

  • Parameters: leave empty.

Script code executed when the data is saved:

import org.cyclos.entities.banking.ScheduledPayment
import org.cyclos.model.ValidationException
import org.cyclos.model.banking.transactions.ScheduledPaymentStatus

import groovy.transform.TypeChecked

@TypeChecked
def closeLoan() {
    ScheduledPayment transaction = binding.variables.transaction as ScheduledPayment
    if (transaction.status == ScheduledPaymentStatus.CANCELED) {
        // Should never cancel a loan scheduled payment
        throw new ValidationException("Cannot cancel a loan")
    } else if (transaction.status == ScheduledPaymentStatus.CLOSED) {
        // Close the loan
        (binding.variables.loan as Loan).close(transaction)
    }
}

closeLoan()

Also, create another script for the authorization extension point, which will create the repayment scheduled payment once the loan is authorized:

  • Name: Loan authorization;

  • Type: Extension point;

  • Included libraries: Loan;

  • Parameters: leave empty.

Script code executed when the data is saved:

import org.cyclos.entities.banking.Payment

import groovy.transform.TypeChecked

@TypeChecked
def createRepayment() {
    Payment transaction = binding.variables.transaction as Payment
    if (transaction.getTransfer() != null) {
        // The transaction was authorized, create the repayment
        (binding.variables.loan as Loan).createRepayment(transaction)
    }
}

createRepayment()
Create the custom operation

Under System > Tools > Custom operations, create a new one, with the following characteristics:

  • Name: Grant loan (can be changed, is the label displayed to users);

  • Enabled: Yes;

  • Scope: User;

  • Script: Grant loan;

  • Script parameters: leave empty;

  • Result type: Notification;

  • Has file upload: No;

  • Main menu: Banking;

  • User management section: Banking;

  • Information text: you can add here some text explaining the process – it will be displayed in the operation page;

  • Confirmation text: add here some text which will be displayed in a confirmation dialog before granting the loan.

After saving, create the following fields:

  • Amount:

    • Internal name: amount;

    • Data type: Decimal;

    • Required: Yes.

  • Installment count:

    • Internal name: numberOfInstallments;

    • Data type: Integer;

    • Required: Yes;

  • First due date:

    • Internal name: firstDueDate;

    • Data type: Date;

    • Required: Yes.

Create the extension points

Under System > Tools > Extension points, create a two new extension points, each with the following characteristics:

  • A transaction extension point (will close the loan when all installments are processed):

    • Name: Close loan;

    • Type: Transaction;

    • Enabled: Yes;

    • Transfer types: Units account – Loan repayment (choose the loan repayment type);

    • Events: Change status;

    • Script: Loan closing;

    • Script parameters: Leave empty.

  • An authorization extension point (will create the repayment once the loan is authorized):

    • Name: Loan authorization;

    • Type: Authorization;

    • Enabled: Yes;

    • Transfer types: Debit account – Loan grant (choose the loan grant type);

    • Events: Authorize;

    • Script: Loan authorization;

    • Script parameters: Leave empty.

Grant the administrator permissions

Under System > User configuration > Groups, select the Network administrators group (or the ones that will grant loans). Then, in the Permissions tab:

  • Under User management > Run custom operations over users, check the Grant loan operation;

  • Under Accounts > Transfer status flows, make Loan visible, but not editable;

  • Under Accounts > Visible transaction fields, select all related custom fields;

  • Under User accounts > Scheduled payments, select View (and maybe process installment and settle too).

Enable the custom operation for users which will be able to receive loans

In System > User configuration > Products (permissions), select the member product for users which will be able to receive loans. In the Custom operations field, make the Grant loan operation enabled. Leave the run checkbox unchecked (or users would be able to grant loans to themselves!).

You can permit users to repay loan installments anticipated in Units. For this you have to check in the member product 'process installment' and the user needs to have permissions to make a payment of the transaction type used for the loan repayments.

4.5.3. Record management

This solution lets you search, view, create, edit and remove (CRUD) records using custom operations. It uses records for being already available as a customizable data storage in Cyclos. But the example can be expanded to other use cases as well, such as custom database tables, or data in an external system. Also, with this functionality it is possible to allow users to view other user’s records, which is not available in Cyclos for records, but users can run operations over other users.

The example record type has just 2 fields: title and description.

To configure this solution, follow carefully each of the following steps:

Create the user record type to work with and give permissions

Under System > System configuration > Record types, create a new 'User record type', with the following characteristics:

  • Name: Daily note;

  • Internal name: dailyNote;

  • Display style: List;

  • Main menu: Personal;

  • User management section: User management.

For this record type, create the following fields:

  • Title:

    • Internal name: title;

    • Data type: Single line text;

    • Required: Yes;

    • Show in results: Yes.

  • Description:

    • Internal name: description;

    • Data type: Multiple line text;

    • Required: No;

    • Show in results: Yes.

Now give permissions:

  • To user group (System > User configuration > Products > Product): in Records, check 'Enable' over 'Daily note'.

Create the library script

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: Records library;

  • Type: Library.

Script code:

import org.cyclos.entities.users.Record
import org.cyclos.entities.users.User
import org.cyclos.entities.users.UserRecord
import org.cyclos.impl.access.SessionData
import org.cyclos.impl.system.ScriptHelper
import org.cyclos.impl.users.RecordServiceLocal
import org.cyclos.impl.utils.formatting.FormatterImpl

class RecordProjection {
    Long recordId
    String title
    String description
    String user
}

class RecordView {
    String type
    String user
    String creationDate
    String lastModifiedDate
    String modifiedBy
    String createdBy
    String title
    String description
}

class RecordHelper {
    Map<String, Object> formParameters
    User user
    ScriptHelper scriptHelper
    SessionData sessionData
    FormatterImpl formatter
    Map<String, String> scriptParameters
    String typeInternalName
    String titleInternalName
    String descriptionInternalName
    RecordServiceLocal recordService

    RecordHelper(Binding binding) {
        formParameters = binding.formParameters
        user = binding.user
        scriptHelper = binding.scriptHelper
        sessionData = binding.sessionData
        formatter = binding.formatter
        scriptParameters = binding.scriptParameters
        typeInternalName = scriptParameters.recordType
        recordService = binding.recordService
    }

    RecordProjection toProjection(Record record) {
        def fields = scriptHelper.wrap(record);
        def user = record instanceof UserRecord ? record.user : null
        return new RecordProjection(
                recordId: record.id,
                user: formatter.format(user),
                title: fields.title,
                description: fields.description)
    }

    RecordView toView(Record record) {
        def fields = scriptHelper.wrap(record);
        def user = record instanceof UserRecord ? record.user : null
        return new RecordView(
                type: formatter.format(record.type),
                user: formatter.format(user),
                creationDate: formatter.format(record.creationDate),
                lastModifiedDate: formatter.format(record.lastModifiedDate),
                modifiedBy: formatter.format(record.modifiedBy),
                createdBy: formatter.format(record.createdBy),
                title: fields.title,
                description: fields.description)
    }
}

It will also needs a single parameter. Copy the following to the 'Script parameters' field, adjusting the internal name in case you set another one:

recordType=dailyNote
Create a custom operation script to send the record by email

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: Send record by email;

  • Type: Custom operation.

  • Run with all permissions: Yes;

  • Included libraries: Records library.

Script code executed when the custom operation is executed:

import org.cyclos.entities.users.UserRecord
import org.cyclos.impl.utils.notifications.MailHandler
import org.springframework.mail.javamail.MimeMessageHelper

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field RecordHelper helper = new RecordHelper(binding)
@Field MailHandler mailHandler = binding.mailHandler

@TypeChecked
def sendByEmail() {
    def id = helper.scriptHelper.unmaskId(helper.formParameters.recordId)
    def record = helper.recordService.find(id) as UserRecord
    def view = helper.toView(record)
    def sender = mailHandler.mailSender
    def message = sender.createMimeMessage()
    def helper = new MimeMessageHelper(message)

    def body = """
        This is your record details:<br>
        <b>Title:</b> ${view.title}<br>
    """
    if (view.description) {
        body += "<b>Description:</b> <pre>${view.description}</pre>"
    }

    mailHandler.send(record.user.name, record.user.email, "Your record", body)

    return "The email was sent"
}
return sendByEmail()
Create the custom operation script to search records

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: Search records;

  • Type: Custom operation;

  • Run with all permissions: Yes;

  • Included libraries: Records library.

Script code executed when the custom operation is executed:

import org.cyclos.impl.search.RecordSearchHandler
import org.cyclos.model.users.records.UserRecordQuery
import org.cyclos.model.users.recordtypes.RecordTypeVO
import org.cyclos.model.users.users.UserVO

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field RecordHelper helper = new RecordHelper(binding)
@Field Integer currentPage = binding.currentPage
@Field Integer pageSize = binding.pageSize
@Field Boolean skipTotalCount = binding.skipTotalCount

@TypeChecked
def searchRecords() {
    def recordSearchHandler = binding.variables.recordSearchHandler as RecordSearchHandler

    def query = new UserRecordQuery()
    def formParameters = helper.formParameters
    if (formParameters.searchOnlyInMyRecords) {
        query.user = new UserVO(helper.user.id)
    }
    query.currentPage = currentPage
    query.pageSize = pageSize
    query.skipTotalCount = skipTotalCount
    query.type = new RecordTypeVO(internalName: helper.typeInternalName)
    query.keywords = formParameters.keywords as String

    def page = recordSearchHandler.searchEntities(query)
    def rows = page.pageItems.collect { helper.toProjection(it) }

    return [
        columns: [
            [header: "Owner", property: "user",width:"15%"],
            [header: "Title", property: "title", width:"35%"],
            [header: "Description", property: "description",width:"50%"]
        ],
        rows: rows,
        totalCount: page.totalCount,
        hasNextPage: page.hasNextPage
    ]
}

return searchRecords()
Create the custom operation script to create records

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: Create record;

  • Type: Custom operation;

  • Run with all permissions: Yes;

  • Included libraries: Records library.

Script code executed when the custom operation is executed

import org.cyclos.model.users.records.RecordDataParams
import org.cyclos.model.users.recordtypes.RecordTypeVO
import org.cyclos.model.utils.NotificationLevel
import org.springframework.transaction.TransactionStatus

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field RecordHelper helper = new RecordHelper(binding)

@TypeChecked
def createRecord() {
    def msg = "The record was created successfully."
    def error = false

    try {
        def data = helper.recordService.getDataForNew(new RecordDataParams(
                recordType: new RecordTypeVO(internalName: helper.typeInternalName)))
        def fields = helper.scriptHelper.wrap(data.dto)
        fields.title = helper.formParameters.title
        fields.description = helper.formParameters.description
        helper.recordService.save(data.dto)
    } catch (Exception ex) {
        // Mark the transaction to be rolled-back
        def transactionStatus = helper.scriptParameters.transactionStatus as TransactionStatus
        transactionStatus.setRollbackOnly()
        error = true
        msg = """There was an error trying to create the record.
            Please, contact the administration."""
    }

    return [
        notification: msg,
        notificationLevel: error ?  NotificationLevel.ERROR : NotificationLevel.INFORMATION,
        backTo: error ? null : "searchRecords",
        reRun: !error
    ]
}

return createRecord()
Create the custom operation script to view records

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: View record;

  • Type: Custom operation;

  • Run with all permissions: Yes;

  • Included libraries: Records library.

Script code executed when the custom operation is executed:

import org.cyclos.entities.users.UserRecord
import org.cyclos.entities.users.UserRecordType
import org.cyclos.utils.StringHelper

import groovy.transform.Field
import groovy.transform.TypeChecked
import groovy.xml.MarkupBuilder

class Styles {
    static String infoBox = "white-space: normal; text-overflow: ellipsis;"
    static String fieldContainerLabel = "white-space: normal; text-overflow: ellipsis; font-weight: 400; font-size: 15px; color: #1865a3;"
    static String fieldContainerValue = "white-space: normal; text-overflow: ellipsis;  margin: 2px 0 9px 0; font-size: 16px; line-height: 18px;"
}

@Field RecordHelper helper = new RecordHelper(binding)

def createContent(StringWriter out, UserRecord record) {
    def view = helper.toView(record)
    UserRecordType type = record.type
    def html = new MarkupBuilder(out)
    html.div(style:"${Styles.infoBox}") {
        div {
            div(style:"${Styles.fieldContainerLabel}") { mkp.yield "Owner:" }
            div(style:"${Styles.fieldContainerValue}") {
                mkp.yield view.user
            }
        }
        if (type.showUpdateToUsers) {
            div {
                div(style:"${Styles.fieldContainerLabel}") { mkp.yield "Created by:" }
                div(style:"${Styles.fieldContainerValue}") {
                    mkp.yield view.createdBy
                }
            }
            div {
                div(style:"${Styles.fieldContainerLabel}") { mkp.yield "Creation date:" }
                div(style:"${Styles.fieldContainerValue}") {
                    mkp.yield view.creationDate
                }
            }
            if (!StringHelper.isBlank(record.lastModifiedDate)) {
                div {
                    div(style:"${Styles.fieldContainerLabel}") {
                        mkp.yield "Last modification date:"
                    }
                    div(style:"${Styles.fieldContainerValue}") {
                        mkp.yield view.lastModifiedDate
                    }
                }
                div {
                    div(style:"${Styles.fieldContainerLabel}") {
                        mkp.yield "Last modification by:"
                    }
                    div(style:"${Styles.fieldContainerValue}") {
                        mkp.yield view.modifiedBy
                    }
                }
            }
        }
        if (!StringHelper.isBlank(view.title)) {
            div {
                div(style:"${Styles.fieldContainerLabel}") { mkp.yield "Title:" }
                div(style:"${Styles.fieldContainerValue}") { mkp.yield view.title }
            }
        }
        if (!StringHelper.isBlank(view.description)) {
            div {
                div(style:"${Styles.fieldContainerLabel}") { mkp.yield "Description:" }
                div(style:"${Styles.fieldContainerValue}") {
                    pre {
                        mkp.yield view.description
                    }
                }
            }
        }
    }
}

@TypeChecked
def viewRecord() {
    def sessionData = helper.sessionData
    def id = helper.scriptHelper.unmaskId(helper.formParameters.recordId)
    def record = helper.recordService.find(id) as UserRecord
    def out = new StringWriter()
    createContent(out, record)

    def loggedInAsOwner = record.user == sessionData.loggedUser
    return [
        content: out.toString(),
        actions: [
            removeRecord: [
                enabled: loggedInAsOwner
            ],
            updateRecord: [
                enabled: loggedInAsOwner
            ]
        ]
    ]
}

return viewRecord()
Create the custom operation script to update records

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: Update record;

  • Type: Custom operation;

  • Run with all permissions: Yes;

  • Included libraries: Records library.

Script code executed when the custom operation is executed:

import org.cyclos.model.users.records.RecordData
import org.cyclos.model.utils.NotificationLevel
import org.springframework.transaction.TransactionStatus

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field RecordHelper helper = new RecordHelper(binding)
@Field Map<String, Object> formParameters = binding.formParameters
@Field TransactionStatus transactionStatus = binding.transactionStatus

@TypeChecked
def updateRecord() {
    def msg = "The record was updated successfully."
    def error = false

    try {
        def id = helper.scriptHelper.unmaskId(formParameters.recordId)
        def data = helper.recordService.getData(id) as RecordData
        def fields = helper.scriptHelper.wrap(data.dto)
        fields.title = formParameters.title
        fields.description = formParameters.description
        helper.recordService.save(data.dto)
    } catch (Exception ex) {
        error = true
        transactionStatus.setRollbackOnly()
        msg = """There was an error trying to update the record.
            Please, contact the administration."""
    }

    return [
        notification: msg,
        notificationLevel: error ?  NotificationLevel.ERROR : NotificationLevel.INFORMATION,
        backTo: error ? null : "recordDetails",
        reRun: !error
    ]
}

return updateRecord()

Script code executed before the form is show, to fill the initial field values:

import org.cyclos.entities.users.UserRecord
import org.cyclos.impl.utils.persistence.EntityManagerHandler
import org.cyclos.utils.StringHelper

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field RecordHelper helper = new RecordHelper(binding)
@Field Map<String, Object> formParameters = binding.formParameters
@Field EntityManagerHandler entityManagerHandler = binding.entityManagerHandler

@TypeChecked
def loadRecordFields() {
    def id = helper.scriptHelper.unmaskId(formParameters.recordId)
    def record = entityManagerHandler.find(UserRecord, id)
    def newRecord = helper.recordService.getData(id).dto
    def view = helper.toView(record)

    return [
        title: StringHelper.emptyIfNull(view.title),
        description: StringHelper.emptyIfNull(view.description)
    ]
}

return loadRecordFields()
Create the custom operation script to remove records

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: Remove record;

  • Type: Custom operation;

  • Run with all permissions: Yes;

  • Included libraries: Records library.

Script code executed when the custom operation is executed:

import org.cyclos.model.utils.NotificationLevel
import org.springframework.transaction.TransactionStatus

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field RecordHelper helper = new RecordHelper(binding)

@TypeChecked
def removeRecord() {
    def msg = "The record was removed successfully."
    def error = false

    def recordId = helper.scriptHelper.unmaskId(helper.formParameters.recordId)
    try {
        helper.recordService.remove(recordId)
    } catch (Exception ex) {
        def transactionStatus = helper.scriptParameters.transactionStatus as TransactionStatus
        transactionStatus.setRollbackOnly()
        error = true
        msg = """There was an error trying to remove the record.
            Please, contact the administration."""
    }

    return [
        notification: msg,
        notificationLevel: error ? NotificationLevel.ERROR : NotificationLevel.INFORMATION,
        backTo: error ? null : "searchRecords",
        reRun: !error
    ]
}

return removeRecord()
Create the custom operation to remove records

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Remove record;

  • Internal name: removeRecord;

  • Enable for channels: Main, Web services, Mobile app;

  • Custom submit label: Remove (can be changed - will be the label displayed in the action button);

  • Scope: Internal;

  • Script: Remove record;

  • Result type: Notification;

  • Custom script execute message: This record will be removed. Do you want to continue? (can be edited - it is necessary to alert the user that the record is going to be removed).

Once saved, on the Form fields tab, create a new field, with the following characteristics:

  • Display name: Record id;

  • Internal name: recordId;

  • Data type: Single line text.

Create the custom operation to update records

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Update record;

  • Internal name: updateRecord;

  • Enable for channels: Main, Web services, Mobile app;

  • Custom submit label: Update (can be changed - will be the label displayed in the action button);

  • Scope: Internal;

  • Script: Update record;

  • Result type: Notification;

  • Show form: Always.

Once saved, on the Form fields tab, create the following fields:

  • Record id:

    • Display name: Record id;

    • Internal name: recordId;

    • Data type: Single line text.

  • Title:

    • Display name: Title;

    • Internal name: title;

    • Data type: Single line text;

    • Required: Yes.

  • Description:

    • Display name: Description;

    • Internal name: description;

    • Data type: Multiple line text.

Create the custom operation to send the record email

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Send record email;

  • Internal name: sendRecordEmail;

  • Enable for channels: Main, Web services, Mobile app;

  • Custom submit label: Send email (can be changed - will be the label displayed in the action button);

  • Scope: Internal;

  • Script: Send record email;

  • Result type: Notification.

Once saved, on the Form fields tab, create the following fields:

  • Record id:

    • Display name: Record id;

    • Internal name: recordId;

    • Data type: Single line text.

Create the custom operation to view records details

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Record details;

  • Internal name: recordDetails;

  • Enable for channels: Main, Web services, Mobile app;

  • Scope: Internal;

  • Script: View record;

  • Result type: Rich text.

Once saved, on the Form fields tab, create the following fields:

  • Record id:

    • Display name: Record id;

    • Internal name: recordId;

    • Data type: Single line text.

Once saved, on the Actions tab, add the following actions:

  • Remove record:

    • Parameters:

      • Record id: Record id.

  • Update record:

    • Parameters:

      • Record id: Record id;

      • Title: Not used;

      • Description: Not used.

  • Send record email.

    • Record id: Record id.

Create the custom operation to create records

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Create record;

  • Internal name: createRecord;

  • Enable for channels: Main, Web services, Mobile app;

  • Label: New (can be changed - will be the label displayed in the action button);

  • Scope: Internal;

  • Script: Create record;

  • Result type: Notification;

  • Show form: Always.

Once saved, on the Form fields tab, create the following fields:

  • Title:

    • Display name: Title;

    • Internal name: title;

    • Data type: Single line text.

  • Description:

    • Display name: Description;

    • Internal name: description;

    • Data type: Multiple line text.

Create the custom operation to search records

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: Daily notes (can be changed - will be the label displayed on the menu);

  • Internal name: searchRecords;

  • Enable for channels: Main, Web services, Mobile app;

  • Scope: User;

  • Script: Search records;

  • Result type: Result page;

  • Allow printing results: Yes;

  • Allow exporting results to CSV: Yes;

  • Action when clicking a row: Run an internal custom operation;

  • Custom operation: Record details;

  • Parameters to be passed (comma-separated names): recordId;

  • Main menu: Personal (can be changed - will be the menu where the label is going to be displayed);

  • User management section: User management (can be changed - will be the section where the label is going to be displayed);

  • Enable for active users: Yes.

Once saved, on the Form fields tab, create the following fields:

  • Keywords:

    • Internal name: keywords;

    • Data type: Single line text.

  • Search only in my records:

    • Internal name: searchOnlyInMyRecords;

    • Data type: Boolean.

Once saved, on the Actions tab, add the following actions:

  • Create record:

    • Visibility: Before and after run the custom operation;

    • Parameters:

      • Title: Not used;

      • Description: Not used.

Enable the custom operation for users

In System > User configuration > Products (permissions), select the member product for users which will be able to work with this operation. In the Custom operations field, make sure the Daily notes is both 'Enabled' and allowed to 'Run over self'.

You can also grant the operation for admins over users.

4.5.4. User balances

This solution provides a custom operation for administrators to search the users' balances, with 2 advantages over the regular user balances overview in Cyclos:

  • It is possible to select a date, so all presented balances will be for that date;

  • The available balances are also shown, but only if no date is set (as credit limit could have changed over time).

These options are not available in the regular balances search because they need to be calculated per user, whereas the current balance is stored in the database. This makes the script unviable when there are too many users. Still, some systems require this functionality.

However, as a drawback, the filters for users are also more limited in this script: it is only possible to filter by a specific group. So, if only the current balance is desired, it is advised to use the built-in functionality instead.

To configure this functionality, follow carefully each of the following steps:

Create the script to load user account types

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: User account types loader;

  • Type: Load custom field values.

Script code that returns the possible values when either creating or editing an entity:

import java.util.stream.Collectors

import org.cyclos.impl.access.SessionData
import org.cyclos.impl.banking.AccountTypeServiceLocal
import org.cyclos.model.banking.accounttypes.AccountTypeNature
import org.cyclos.model.system.fields.DynamicFieldValueVO

import groovy.transform.TypeChecked

@TypeChecked
def loadUserAccountTypes() {
    def variables = binding.variables
    def accountTypeService = variables.accountTypeService as AccountTypeServiceLocal
    def sessionData = variables.sessionData as SessionData
    return sessionData.getProducts().grantedAccountTypes()
            .stream()
            .filter { it.getNature() == AccountTypeNature.USER }
            .map { new DynamicFieldValueVO(String.valueOf(it.id), it.name) }
            .collect(Collectors.toList())
}

loadUserAccountTypes()
Create the script to load user groups

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: User groups loader;

  • Type: Load custom field values;

Script code that returns the possible values when either creating or editing an entity:

import java.util.stream.Collectors

import org.cyclos.impl.access.SessionData
import org.cyclos.impl.users.GroupsHandler
import org.cyclos.model.system.fields.DynamicFieldValueVO

import groovy.transform.TypeChecked

@TypeChecked
def loadUserGroups() {
    def variables = binding.variables
    def groupsHandler = variables.groupsHandler as GroupsHandler
    def sessionData = variables.sessionData as SessionData

    return groupsHandler
            .getAccessibleUserGroups(sessionData.getLoggedUser())
            .stream()
            .map { new DynamicFieldValueVO(String.valueOf(it.id), it.name) }
            .collect(Collectors.toList())
}

loadUserGroups()
Create the custom operation script to search users balances

Under System > Tools > Scripts, create the next script, with the following characteristics:

  • Name: User balances;

  • Type: Custom operation;

  • Run with all permissions: Yes.

Script code executed when the custom operation is executed:

import java.util.stream.Collectors

import org.cyclos.entities.banking.QAccount
import org.cyclos.entities.banking.QAccountType
import org.cyclos.entities.banking.UserAccountType
import org.cyclos.entities.system.DynamicFieldValue
import org.cyclos.entities.system.ExportFormat
import org.cyclos.entities.users.QGroup
import org.cyclos.entities.users.QUser
import org.cyclos.entities.users.User
import org.cyclos.entities.users.UserCustomField
import org.cyclos.impl.ApplicationHandler
import org.cyclos.impl.InvocationContext
import org.cyclos.impl.access.SessionData
import org.cyclos.impl.banking.AccountServiceLocal
import org.cyclos.impl.contentmanagement.TranslationHandler
import org.cyclos.impl.system.ScriptHelper
import org.cyclos.impl.users.UserCustomFieldServiceLocal
import org.cyclos.impl.utils.formatting.FormatterImpl
import org.cyclos.impl.utils.persistence.EntityManagerHandler
import org.cyclos.model.ValidationException
import org.cyclos.model.banking.BankingKeys
import org.cyclos.model.users.UsersKeys
import org.cyclos.server.utils.DateHelper

import com.querydsl.core.types.Expression
import com.querydsl.core.types.dsl.Expressions

import groovy.transform.Field
import groovy.transform.TypeChecked

@Field EntityManagerHandler entityManagerHandler = binding.entityManagerHandler
@Field TranslationHandler translationHandler = binding.translationHandler
@Field AccountServiceLocal accountService = binding.accountService
@Field FormatterImpl formatter = binding.formatter
@Field ExportFormat exportFormat = binding.exportFormat
@Field UserCustomFieldServiceLocal userCustomFieldService = binding.userCustomFieldService
@Field ApplicationHandler applicationHandler = binding.applicationHandler
@Field ScriptHelper scriptHelper = binding.scriptHelper
@Field Map<String, Object> formParameters = binding.formParameters
@Field SessionData sessionData = binding.sessionData
@Field Integer currentPage = binding.currentPage
@Field Integer pageSize = binding.pageSize
@Field Boolean skipTotalCount = binding.skipTotalCount

@TypeChecked
def searchBalances(){
    def a = QAccount.account
    def at = QAccountType.accountType
    def u = QUser.user
    def g = QGroup.group

    def accountTypeId = (formParameters.accountType as DynamicFieldValue)?.value as Long
    def accountType = entityManagerHandler.find(UserAccountType, accountTypeId)

    def query = entityManagerHandler
            .from(u)
            .innerJoin(g).on(u.group().eq(g))
            .leftJoin(a).on(a.user().eq(u))
            .leftJoin(at).on(a.type().eq(at))

    def groups = (formParameters.groups as Collection<DynamicFieldValue>)
    if (groups && !groups.empty) {
        def groupIds = groups.collect {
            Long.valueOf(it.value)
        }
        query.where(g.id.in(groupIds))
    }

    query
            .where(at.isNull().or(at.eq(accountType)))
            .limit(pageSize)
            .offset(pageSize * currentPage)
            .orderBy(u.name.asc(), at.name.asc())

    def totalCount = skipTotalCount ? null : query.fetchCount()

    def balanceExpression
    def expressions = [
        u.id,
        u.displayForManagers,
        at.name,
        at.currencyId
    ] as List<Expression>
    def date = formParameters.date as Date
    if (date) {
        // Validate the archiving date, if any
        def archivingDate = applicationHandler.application.archivingDate
        if (archivingDate && date.before(archivingDate)) {
            throw new ValidationException(
            "Account data before ${formatter.format(archivingDate)} is archived")
        }

        query.where(a.creationDate.before(date))
        def timeZone = sessionData.getConfiguration().getTimeZone()

        // When there's a date, get the balance at that particular date
        date = DateHelper.shiftToEnd(date, timeZone)
        balanceExpression = a.balance(Expressions.constant(date))
        expressions << balanceExpression
    }

    List<UserCustomField> customFields = []
    if (exportFormat && exportFormat.internalName != 'pdf') {
        // When exporting tabular data, include the profile fields
        customFields = userCustomFieldService.listAll().findAll() {
            it.includeInExport
        }
    }

    def cacheFlusher = InvocationContext.newCacheFlusher()
    def rows = query.stream(expressions as Expression[]).map {
        Long userId = it.get(u.id)
        def result = [
            id: userId,
            display: it.get(u.displayForManagers),
            accountType: accountType.name,
            currency: accountType.currency.id
        ] as Map<String, Object>
        // We need to fetch the user for current date or custom fields
        User user = null
        if (date === null || !customFields.empty) {
            user = entityManagerHandler.find(User, userId)
        }
        if (date == null) {
            // When no date, we get the current account status, to fetch the available balance
            def account = accountService.load(user, accountType)
            def status = accountService.getAccountStatus(account, null, null)
            result.balance = status.balance
            result.availableBalance = status.availableBalance
        } else {
            // The balance is an expression for a specific date
            result.balance = it.get(balanceExpression)
        }
        if (!customFields.empty) {
            def fields = scriptHelper.wrap(user, customFields)
            customFields.each {
                def value = fields[it.internalName]
                result[it.internalName] = value instanceof Date ?
                        formatter.formatAsDate(value) : formatter.format(value)
            }
        }
        cacheFlusher.flush()
        return result
    }.collect(Collectors.toList())

    def columns = [
        [
            header: translationHandler.message(UsersKeys.Users.USER),
            property: "display"
        ],
        [
            header: translationHandler.message(BankingKeys.Accounts.TYPE),
            property: "accountType"
        ],
        [
            header: translationHandler.message(BankingKeys.Accounts.BALANCE),
            property: "balance",
            currencyProperty: "currency",
            align: "right"
        ]
    ]
    if (date == null) {
        columns << [
            header: translationHandler.message(BankingKeys.Accounts.AVAILABLE_BALANCE),
            property: "availableBalance",
            currencyProperty: "currency",
            align: "right"
        ]
    }
    customFields.each {
        columns << [ header: it.name, property: it.internalName ]
    }
    return [
        columns: columns,
        rows: rows,
        totalCount: totalCount,
        currentPage: currentPage
    ]
}

searchBalances()
Create the custom operation to search users balances

Under System > Tools > Custom operations, create a new one with the following characteristics:

  • Name: User balances (can be changed - will be the label displayed on the menu);

  • Internal name: userBalances;

  • Enable for channels: Main, Web services, Mobile app;

  • Scope: System;

  • Script: User balances;

  • Result type: Result page;

  • Search automatically on page load: No;

  • Allow printing results: Yes;

  • Allow exporting results to CSV: Yes;

  • Action when clicking a row: Navigate to a Cyclos location;

  • Location: user_profile;

  • Parameters to be passed (comma-separated names): id;

  • Main menu: Banking (can be changed - will be the menu where the label is going to be displayed).

Once saved, on the Form fields tab, create the following fields:

  • Account type:

    • Internal name: accountType;

    • Data type: Dynamic selection;

    • Required: true;

    • All selected label: All;

    • Load values script: User account types loader.

  • Groups:

    • Internal name: groups;

    • Data type: Dynamic multi selection;

    • All selected label: All (can be changed as desired);

    • Load values script: User groups loader.

  • Date:

    • Internal name: date;

    • Data type: Date.

Enable the custom operation

Enable this operation in the administrators group, in the Permissions tab, under 'Run system custom operations'.

4.6. Script storage

A general-purpose storage is available for scripts. It is a key / value storage, implementing the ObjectParameterStorage interface.

It stores the values as JSON in the database. Besides the methods for get / set String, Boolean, Decimal, Integer, Long and Enum, it also supports storing objects.

Also, a mechanism is provided for Groovy scripts to access objects directly via the property name, such as storage.name = value or value = storage.name.

A script storage is obtained using a key (string), and a timeout can (optionally) be set before the storage expires. The storage is accessed via the ScriptStorageHandler. It provides the following methods:

  • get(key) or get(key, timeoutInSeconds): Returns a storage by key (string). If a valid storage exists (in the same network), it is returned. Otherwise, a new one is created and returned. Optionally, a timeout in seconds can be passed, which sets an expiration for the stored data;

  • exists(key): Returns whether a valid storage with a given key exists;

  • remove(key): Removes a storage by key;

  • detach(key): Detaches the given storage from the current transaction. Subsequent calls to get will re-read the data from the database.

Some restrictions apply on which kind of objects can be stored or retrieved. Entities can only be stored if they are already persisted (only the id is stored, and the entity is loaded by id from the database when retrieved). Other objects need to have a public empty constructor, plus getters and setters for fields.

Here is an example. It will also cover the case where in the first request of a user, a complex computation is performed. And we want to make sure this is done exactly once per user. So the example is a bit complex, but it prevents any race conditions.

// Storage retrieval
def timeout = 60 * 60 * 3 // Expires in 3 hours
def key = "requests_for_${sessionData.loggedUser.id}"
def storage = scriptStorageHandler.get(key, timeout)

def currentRequests = storage.requests ?: 0

// If the user has never done any request, we'll do some unique and costly operation...
if (currentRequests == 0) {
    // But we need to make sure that no other thread can make this code run twice
    lockHandler.lock(key)
    // After locking, detach the storage to force it to be re-read
    scriptStorageHandler.detach(key)

    // Re-read the storage in the critical section (after lock)
    storage = scriptStorageHandler.get(key)
    currentRequests = storage.requests ?: 0

    // We need to check again if this is still the first request
    if (currentRequests == 0) {
        // Only now we're 100% sure that this is the first request for the user
        // Do the complex computation here...
        // We know this is the user's first request, so store 1
        storage.requests = 1
    } else {
        // Increment normally, as the initial computation was already done by another transaction
        storage.requests++
    }
} else {
    // Increment the number of requests for the logged user
    storage.requests++
}

// Then, later on, maybe on another script...
return "There are ${storage.requests} requests for user ${sessionData.loggedUser.name}"

4.7. Custom alerts

From Cyclos 4.12 onwards, custom alerts can be defined for both system and user. These alerts will be created using scripting. It can be done in any kind of script just adding a few lines as the next examples show:

  • System alert

import org.cyclos.impl.messaging.AlertServiceLocal
import org.cyclos.model.messaging.alerts.SystemAlertType

AlertServiceLocal alertService = binding.alertService
alertService.create(SystemAlertType.CUSTOM, /*Alert text*/ "This is a system alert example")
  • User alert

import org.cyclos.impl.messaging.AlertServiceLocal
import org.cyclos.model.messaging.alerts.UserAlertType

AlertServiceLocal alertService = binding.alertService;
// Create the alert for the corresponding user, in this case we use the one logged.
alertService.create(sessionData.getLoggedUser(), UserAlertType.CUSTOM, /*Alert text*/ "This is a user alert example")

This type will appear as an option in the administrator’s notification settings page under User alerts and System alerts preferences.

4.8. Debugging scripts

The script editor in Cyclos uses CodeMirror, which provides syntax highlighting. However, as the script complexity grows, better tooling support is desired. We use and support using Eclipse, together with the Groovy plugin.

For this purpose, Cyclos provides, in the details page of scripts, a button named Get code for debug. It will download a ZIP file with the full code of each script box, with all library code already included (there are comments separating each include). Also, the script parameters are returned, including script parameters of included libraries as well. This is the code that is actually executed by Cyclos.

In order to run the scripts in the IDE, a local copy of the Cyclos database is needed, as well as a copy of the JARs bundled with Cyclos. Also, a small Groovy script needs to be written to provide the actual script the context (bound variables). The ZIP file also includes a README.html file, which will explain how to set up the environment and run the script.

5. Content management

Cyclos allows customizing content in several ways, such as:

  • Static content: Headers, footers, home page, etc.;

  • Menu pages: Content pages which is shown as a menu item;

  • Mobile pages: Content pages shown in the mobile application;

  • Themes: Theme used in one of the front-ends (classic web frontend, new web frontend, mobile app);

  • Application translation: Translation of static text displayed in one of the front-ends;

  • Data translation: Translation of different data types, such as profile fields, groups, account types, etc.;

  • Voucher templates: PDF templates for printing vouchers;

  • Banners: Small sections which display custom content in either classic or new frontend;

  • Documents: Either static files or dynamic documents which are applied to users;

  • Images: Logos, custom images to display in static content / pages;

  • SVG icons: Icons used in frontends for custom operations, records, etc.

This chapter covers details on some aspects of content management, which are done by administrators with permission using the classic frontend. It is under the menu 'Content'. Many of the content management concepts are applied to specific configurations. Hence, when accessing the menu for translations, pages and banners, first you need to choose to which configuration will the content be applied to.

5.1. Security considerations

Cyclos allows administrators with either 'System configuration', 'Manage specific configurations' or 'Manage content for configurations' permissions to create and edit HTML pages. Please be aware that JavaScript (and any other HTML tag) can be executed by these pages! Please only grant these permissions to trusted personnel!

Cyclos does not sanitize these pages because it gives administrators more freedom in creating desired content. Script code in <script> tags will not be executed, but scripts on events such as onload will be executed.

5.2. User variables

It is possible to use variables with data of the current user in places like sent notifications, emails and in-app push notifications, as well as static content, content pages, mobile pages and banners.

Variables are always used between curly braces (like {variable}). The list of available variables is:

  • {display}: The configured user’s display name (is the user’s full name by default, but can be changed in the configuration);

  • {name} or {fullName}: The user’s full name;

  • {username}, {loginName} or {login}: The user’s login name;

  • {userEmail} or {email}: The user’s email address. Note: for email templates, use {userEmail} because {email} is the email destination and they could differ;

  • {<internalName>}: A user custom field by internal name;

  • {phone}: A single mobile or land-line phone;

  • {phones}: A comma-separated list with all mobile and land-line phone;

  • {mobilePhone}: A single mobile phone;

  • {mobilePhones}: A comma-separated list with all mobile phones;

  • {landLinePhone}: A single land-line phone;

  • {landLinePhones}: A comma-separated list with all land-line phones;

  • {address}: The formatted user’s default address;

  • {address.name}: The name of the user’s default address;

  • {address.addressLine1}: The line1 of the user’s default address;

  • {address.addressLine2}: The line2 of the user’s default address;

  • {address.street}: The street of the user’s default address;

  • {address.buildingNumber}: The building number of the user’s default address;

  • {address.complement}: The complement of the user’s default address;

  • {address.neighborhood}: The neighborhood of the user’s default address;

  • {address.poBox}: The PO-box of the user’s default address;

  • {address.zip}: The ZIP code of the user’s default address;

  • {address.city}: The city of the user’s default address;

  • {address.region}: The region of the user’s default address;

  • {address.country}: The country name of the user’s default address;

  • {groupSet}: The name of the user’s group set;

  • {group}: The name of the user’s group;

  • {groupDisplay}: The display name at registration for the user’s group;

  • {applicationName}: The application name set in the user’s configuration;

  • {applicationURL}: The application root URL set in the user’s configuration;

  • {date}: The current date, formatted as configured in Cyclos;

  • {dateTime}: The current date and time, formatted as configured in Cyclos;

  • {time}: The current time, formatted as configured in Cyclos;

  • {<typeInternalName>.number}: The user’s account number;

  • {<typeInternalName>.balance}: The formatted user’s account balance;

  • {<typeInternalName>.availableBalance}: The formatted user’s account available balance;

  • {<typeInternalName>.creditLimit}: The formatted user’s account lower credit limit;

  • {<typeInternalName>.upperCreditLimit}: The formatted user’s account upper credit limit;

  • {<typeInternalName>.typeName}: The user’s account type display name;

  • {<typeInternalName>.typeId}: The user’s account type internal id;

Note that for all account-related variables, if the provided account type internal name is account, it will use the first visible account type. For systems with a single user account, it will be easier to use the account prefix.

5.3. Using Thymeleaf for dynamic content

Starting with Cyclos 4.16, in some content elements, the HTML markup can be enhanced with (Thymeleaf)[https://www.thymeleaf.org/] to add dynamic content. These elements are:

  • Static content

  • Menu pages

  • Mobile pages

  • Banners

  • Voucher templates

  • Dynamic documents

Additionally, it is possible to generate dynamic content in both custom operation and custom wizard step information texts.

Note that when using Thymeleaf it is not recommended to edit the content with the visual editor in the content editor. Always prefer the source editor, which allows editing the raw HTML source code, and won’t mess up with the content.

5.3.1. Thymeleaf markup

Thymeleaf uses a minimally-invasive markup over HTML, adding attributes to HTML tags which are processed dynamically. Expressions are evaluated in tag attributes with the ${expressions} syntax. The attributes processed by thymeleaf start with th:, and some of the most used ones are:

  • th:if: Conditionally render the enclosing HTML element. Example:

<div th:if="${user.admin}">This content is only rendered for administrators</div>
  • th:text: Replaces the element content with the result of an expression. The content is automatically escaped to render safe HTML. If the expression value is itself an HTML text, use th:utext instead, which will render the content unescaped. Example:

<div th:text="${user.name}">This content is replaced by the current user's name</div>
  • th:each: Renders the enclosing element many times, one per element in a collection. Example (uses both #account and #format expression objects, which are explained below):

<div th:each="account : ${#accounts.all()}">
    <span th:text="${account.type.name}"></span>:
    <span th:text="${#format.amount(account.currency, account.balance)}"></span>
</div>
  • th:with: Defines a new variable to be used in an expression. Note that you can have a single th:with per element. Example:

<div th:each="account : ${#accounts.all()}" th:with="type=${account.type}"
    th:text="${type.name}"></div>
  • th:block: This is the only element handled by Thymeleaf, and it actually renders no element at all in the resulting HTML, but is useful for conditionals and loops. In the following example, both text blocks will be rendered without any enclosing HTML element:

<th:block th:if="${user.member}">
    You are a regular user.
    <th:block th:if="${user.group.internalName == 'members'}">
        And you are in the members group!
    </th:block>
</th:block>
  • ${{expression}}: Formats a variable for output. It will delegate to the Cyclos built-in formatter, which, is able to format a wide range of data types. It is semantically equivalent to using the ${#format.object(expression)}. Here’s an example which outputs the current date and time with the format taken from Cyclos' configuration:

<th:block th:text="${{now}}"></th:block>

The expressions have access to variables. The variables that Cyclos provides in all contexts are:

  • sessionData: Contains information about the currently authenticated user (if any), as a org.cyclos.impl.access.SessionData;

  • user: The currently authorized user, as a wrapped user, so custom fields are accessible directly. For guests, is null;

  • now: The current date / time as java.util.Date.

Additionally, when processing dynamic documents, there are the following extra attributes:

  • owner: The user over which the document is being processed. Will be different from the currently authorized user when an administrator or broker is printing a user document. It is a wrapped user, so custom fields are accessible directly;

  • customValues: A map with the values of custom fields selected in the form.

5.3.2. Thymeleaf expression utility objects

Thymeleaf offers the concept of expression utility objects, which act as helpers in expressions. There are several built-in objects, please, refer to the documentation for a reference of them.

Besides the built-in expression utility objects, Cyclos provides a set of utility objects:

#format

Handles data formatting. Provides the following methods:

  • object(object): Attempts to format the given input for display. Dates are formatted as date and time;

  • objectOrDate(object): Attempts to format the given input for display. Dates are formatted as date-only;

  • amount(currency, amount): Formats an amount as currency amount. The first parameter is a reference to a currency, or one of: internal name, an entity that has a currency or a currency VO. The second parameter is a number, a currency amount or a string;

  • number(number, scale): Formats a number using a given number of decimals;

  • percentage(number, scale): Formats a number as percentage, using a given number of decimals (scale). 0.1 = 10%, 1 = 100%;

  • objectOrDate(object): Attempts to format the given input for display. Dates are formatted as date-only;

  • url(image): Returns the public API URL for the given image;

  • var(name): Returns a variable for the current user. The available variables are the same as User variables. Just make sure to pass the variable name as string (between quotes), like ${#format.var('mobilePhone')};

  • country(code): Returns a country name from the 2-letter ISO 3166-2 code;

  • truncate(text, length): Truncates a text for a maximum length;

  • maskId(id): Applies the id mask, that means, returns the external representation of a database id.

#decimals

Manipulation and comparison of decimal values. Provides the following methods:

  • decimal(number): Converts the given object into a BigDecimal;

  • areEquals(number1, number2): Returns whether 2 numbers (each one processed by decimal(number)) represent the same decimal amount;

  • isPositive(number): Returns whether the given number (processed by decimal(number)) is positive (and not zero);

  • isPositiveOrZero(number): Returns whether the given number (processed by decimal(number)) is positive or zero;

  • isNegative(number): Returns whether the given number (processed by decimal(number)) is negative;

  • isNegativeOrZero(number): Returns whether the given number (processed by decimal(number)) is negative or zero;

  • isZero(number): Returns whether the given number (processed by decimal(number)) is zero;

  • isNotZero(number): Returns whether the given number (processed by decimal(number)) is not zero.

#accounts

Provides access to the current user accounts. If the current user is an administrator, provides access to visible system accounts. Provides the following methods:

  • all(): Returns all visible accounts;

  • get(type): Returns an account with a given type, using the type internal name or account type reference;

  • first(): Returns the first account. Useful for systems with a single account.

The returned objects are of type org.cyclos.impl.banking.AccountWrapper, which has the following:

#permissions

Allows querying for permissions providing the following methods:

  • has(string): Returns true if the logged user has the given permission. See org.cyclos.model.access.Permission for the list with the valid names. E.g.: 'MY_PAYMENTS_RECEIVE' and 'myPaymentReceive' are two valid names for the same permission.

<div th:if="${#permissions.has('MY_PAYMENTS_RECEIVE')}">
  The user has the receive payment permission
</div>