Automatic Maintenance Service

Last modified by Aurelie Bertrand on 2025/12/19 08:51

DigDash Enterprise provides a maintenance service consisting of :

  • A file cleaner (also known as Files GC) that cleans up all unused files: old history files, cubes and other Flow-dependent files.
  • An automatic configuration backup

File cleaner

The cleaner cleans up files not used by user and role wallets.

The cleaner scans the indexes of all users, as well as the disk, to find files that are no longer linked to the indexes. Identified files are deleted. The files deleted are: cube files (.dcg), cube js files (cube_data_id.js), templates (cube_dm_id.js) and flows (cube_view_id_js).

This operation has the advantage of freeing up disk space and potentially speeding up searches for js files, which can become significant for large volumes (number of personal cubes * number of histories > 100,000).

Depending on the age of the server and the size of the files concerned (number of refreshes carried out, etc.), the operation can take a long time to run for the first time (on some deployments with a lot of users and a lot of customised cubes, one to two hours).

Then, if the clean-up is carried out on a regular basis, the execution time will be shorter. This time depends very much on the performance of the file system and the machine, which makes it difficult to estimate.

Automatic backup

The automatic backup is performed before the files are cleaned. The file generated is copied to the Work Directory (AppData): <digdash.appdata>/Enterprise Server/<domain>/backups/<day's date>.zip.
This backup does not include data sources in order to limit the volume of files.

By default, maintenance is performed every day at midnight.

Important:

By default, the maintenance service only starts if no user session is active at that time. In addition, no user can connect to DigDash Enterprise while it is running. Be careful to program it correctly so that it does not interfere with the normal use of DigDash Enterprise by users or the scheduler. In some cases, we advise you to program the maintenance service at night, and at different times than the scheduler.

Activating, deactivating and/or cleaning at start-up

This section describes how to activate and schedule the maintenance service.
The file cleaner can be activated in one of two ways:

From the Server Status page

The Server Status page can be accessed from the DigDash Enterprise home page by clicking on the Configuration and Server Status buttons.

In the Maintenance service section, click the blue arrow next to Maintenance Service Started to start the cleaner:

Maintenance_service_EN.png

In this example, the next cleanup will take place at midnight. This is the default cleanup time if none is defined in system.xml, by the FILESGC_SCHEDXML parameter. See paragraph Programming and options for automatic maintenance for more details on programming the cleaner.
To start the file cleaner immediately, click on the icon 1700040675471-344.png.

This activation of the maintenance service has a specific behaviour equivalent to the FORCE_FILESMAINTENANCE internal event : the file cleaner is asked to start a cleaning operation. All current sessions are closed. The session manager and scheduler are stopped and restarted once the operation is complete.

From the digdash.properties file

File modified: digdash.properties
See chapter "Externalizing parameters to a properties file"to perform this operation.

Activate or not the Files GC module and/or launch the cleanup at server startup.

ℹ Tasks in progress (scheduled tasks) will cancel the maintenance service. A new attempt will be made 1 hour later. After 5 failed attempts, the service will no longer pass (5 new attempts maximum) and will be postponed until the next scheduled maintenance date.

Available parameters :

  • Name: ddenterpriseapi.startCleaner
    Value: Boolean (default: false)
    Description:
    • true: automatic file cleanup scheduled.
      Note: the cleanup time is defined in system.xml, by the FILESGC_SCHEDXML parameter.
      Thedefault cleanup time (if none is specified in system.xml, FILESGC_SCHEDXML) is every day at 0:00
    • false (default): file cleaner not used
       
  • Name: ddenterpriseapi.cleanOnStart
    Value: Boolean (default: false)
    Description:
    • true: cleans up unused files at server startup (history files, cubes, results files, etc.)
    • false (default): does not clean up unused files at server startup
       
  • Name: ddenterpriseapi.autoBackup
    Value: Boolean (default: false)
    Description:
    • *. true: enables scheduled automatic backup.
    • false (default): does not enable scheduled automatic backup

In the digdash.properties file, all parameters are prefixed with the name of the application concerned.
Here, the default is ddenterpriseapi (ddenterpriseapi.war).
If you have renamed this application, for example to ddapi_dev.war, the parameters become ddapi_dev.startCleaner, ddapi_dev.cleanOnStart, ...

Programming and options for automatic maintenance

File modified: system.xml.

The available parameters are listed below.

FILESGC_SCHEDXML

Value: XML sentence (encoded) (default: none)
Description: This parameter contains an encoded XML sentence describing the scheduling frequency.

Example:

<Property key="FILESGC_SCHEDXML"
value="&lt;Schedule frequency=&quot;daily&quot;
fromDay=&quot;11&quot; fromHour=&quot;0&quot;
fromMinute=&quot;0&quot; fromMonth=&quot;7&quot;
fromYear=&quot;2021&quot; periods=&quot;1&quot;
time=&quot;0:0&quot;/&gt;"></Property>

The attributes are as follows:

  • frequency : hourly, daily, weekly or monthly.
  • fromDay, fromHour, fromMinute, fromMonth, fromYear: start date (❗fromMonth starts at 0 for January). In general, you should enter a date in the past.
  • periods : number of hours, days or months (depending on the selected frequency) between 2 cleanings.
  • time (cleaning time for the daily, weekly and monthly frequencies.

The example above means every day (frequency="daily" and periods="1") at 0:00 (time="0:0").

For a weekly frequency (weekly), we will have an additional weekDays attribute with the following values (to be added together for a trigger on several days of the week. For example, Monday and Wednesday: 2 + 8 = 10):

  • Sunday: 1
  • Monday: 2
  • Tuesday: 4
  • Wednesday: 8
  • Thursday: 16
  • Friday: 32
  • Saturday: 64

For example, for a schedule every 2 weeks, on Mondays and Thursdays, at 18:00, from 1 January 2025 00:00, the property will have the following value:

<Property key="FILESGC_SCHEDXML"
value="&lt;Schedule frequency=&quot;weekly&quot;
fromDay=&quot;1&quot; fromHour=&quot;0&quot;
fromMinute=&quot;0&quot; fromMonth=&quot;0&quot;
fromYear=&quot;2025&quot; periods=&quot;2&quot;
time=&quot;0:0&quot; weekDays=&quot;18&quot;/&gt;">
</Property>

FILESGC_SESSIONSCHECK

Value: true/false (Boolean) (default: none, equivalent to true)
Description: This parameter indicates whether the file cleaner should check for active sessions (connected users) before launching (true), or whether it launches regardless of the status of active sessions (false). In the latter case, all sessions will be disconnected instantly.

Example:

<Property key="FILESGC_SESSIONSCHECK" value="false"></Property>

FILESGC_TASKSCHECK

Value: true/false (Boolean) (default: none, equivalent to true)
Description: This parameter indicates whether the file cleaner should check active tasks before starting (true), or whether it should start regardless of the status of active tasks (false). In the latter case, all tasks will be aborted.

Example:

<Property key="FILESGC_SESSIONSCHECK" value="false"></Property>

💡 Combination of FILESGC_SESSIONSCHECK and FILESGC_TASKSCHECK parameters

  • Case 1 : FILESGC_SESSIONSCHECK = true and FILESGC_TASKCHECK = true (default behaviour).
    ➡ The maintenance service only passes if no user session or task is active.
  • Case 2 : FILESGC_SESSIONCHECK = false and FILESGC_TASKCHECK = true.
    ➡ The maintenance service runs regardless of the status of the sessions but is cancelled if any tasks are active.
  • Case 3 : FILESGC_SESSIONCHECK = false and FILESGC_TASKCHECK = false.
    ➡ The service runs regardless of the status of the sessions and tasks. Active sessions are disconnected and active tasks are stopped.
  • Case 4 : FILESGC_SESSIONCHECK = true and FILESGC_TASKCHECK = false.
    ➡ The maintenance service passes if there are no active sessions (user connected), whatever the status of the tasks. If this is the case, the tasks are interrupted.

USEAUTOBACKUP

Value: true/false (Boolean) (default: none, equivalent to false)
Description: This parameter indicates whether the maintenance service also performs a full configuration backup before executing the file cleanup.

💡 Scheduling automatic backup

Automatic backup is performed according to the schedule defined for automatic maintenance (parameter FILESGC_SCHEDXML) regardless of the status of sessions or tasks. It is therefore possible for automatic backup to be carried out but for file cleanup not to be performed. For example, if the maintenance service checks the active tasks before starting and there are any.
The automatic backup will not be carried out when new attempts are made to clean up the files.

PROP_FILESGC_MIN_FILE_AGE

Value: number (minutes) (default: 0)
Description: This parameter specifies the minimum age in minutes that a file must be to be deleted (if it is to be deleted). If the file is newer than this value then it will not be deleted.

PROP_FILESGC_FASTMODE

Value: true/false (Boolean) (default: none, equivalent to false)
Description: This parameter indicates whether the maintenance service must open a session for each system user (false, the normal case) or whether it can do without (true). If it is enabled, then the scan of files to be cleaned is faster, but there is a risk that some files will not be deleted, for example those of flows and models that have been deleted but still have data on disk.

PROP_FILESGC_MAXTIME

Value: number (minutes) (default: -1, equivalent to no limit)
Description: This parameter specifies the maximum time dedicated to cleaning files. If the time is exceeded, file cleaning is interrupted and there will be files left to clean the next time. Warning: if this time is unreasonably short, the cleaner may do nothing. This value is highly dependent on the Digdash configuration and should be set according to the average cleanup times observed.

PROP_FILESGC_RETRY_MAX_TIME

Value: number (minutes) (default: 360)
Description: This parameter specifies the maximum length of time that the file cleaner will retry cleanup attempts in the event that the cleanup has failed (due to active sessions or tasks).
For example, if the value is set to 360, it will retry for a maximum of 6 hours after the first attempt.

PROP_FILESGC_RETRY_PERIOD

Value: number (minutes) (default: 60)
Description: This parameter specifies the time between 2 file clean-up attempts if the clean-up has failed (due to active sessions or tasks).
For example, if the value is set to 60, it will retry every hour if the previous attempt failed, for the maximum time defined in the PROP_FILESGC_RETRY_MAX_TIME parameter.