Last modified by Aurelie Bertrand on 2025/12/19 08:51
Show last authors
1 {{ddtoc/}}
2
3 ----
4
5 DigDash Enterprise provides a maintenance service consisting of :
6
7 * A file cleaner (also known as Files GC) that cleans up all unused files: old history files, cubes and other {{glossaryReference glossaryId="Glossary" entryId="Flux"}}Flow{{/glossaryReference}}-dependent files.
8 * An automatic configuration backup
9
10 = File cleaner =
11
12 The cleaner cleans up files not used by user and role wallets.
13
14 The cleaner scans the indexes of all users, as well as the disk, to find files that are no longer linked to the indexes. Identified files are deleted. The files deleted are: cube files (.dcg), cube js files (cube_data_id.js), templates (cube_dm_id.js) and flows (cube_view_id_js).
15
16 This operation has the advantage of freeing up disk space and potentially speeding up searches for js files, which can become significant for large volumes (number of personal cubes * number of histories > 100,000).
17
18 Depending on the age of the server and the size of the files concerned (number of refreshes carried out, etc.), the operation can take a long time to run for the first time (on some deployments with a lot of users and a lot of customised cubes, one to two hours).
19
20 Then, if the clean-up is carried out on a regular basis, the execution time will be shorter. This time depends very much on the performance of the file system and the machine, which makes it difficult to estimate.
21
22 = Automatic backup =
23
24 The automatic backup is performed before the files are cleaned. The file generated is copied to the Work Directory (AppData): **<digdash.appdata>/Enterprise Server/<domain>/backups/<day's date>.zip.**
25 This backup does not include data sources in order to limit the volume of files.
26
27 By default, maintenance is performed every day at midnight.
28
29 (% class="box errormessage" %)
30 (((
31 **Important:**
32
33 By default, the maintenance service only starts if no user session is active at that time. In addition, no user can connect to DigDash Enterprise while it is running. Be careful to program it correctly so that it does not interfere with the normal use of DigDash Enterprise by users or the scheduler. In some cases, we advise you to program the maintenance service at night, and at different times than the scheduler.
34 )))
35
36 = Activating, deactivating and/or cleaning at start-up =
37
38 This section describes how to activate and schedule the maintenance service.
39 The file cleaner can be activated in one of two ways:
40
41 == From the Server Status page ==
42
43 The **Server Status** page can be accessed from the DigDash Enterprise home page by clicking on the **Configuration **and **Server Status** buttons.
44
45 In the **Maintenance service** section, click the blue arrow next to **Maintenance Service Started** to start the cleaner:
46
47 [[image:Maintenance_service_EN.png]]
48
49
50 In this example, the next cleanup will take place at midnight. This is the default cleanup time if none is defined in **system.xml**, by the **FILESGC_SCHEDXML** parameter. See paragraph [[Programming and options for automatic maintenance>>doc:||anchor="Options_auto"]] for more details on programming the cleaner.
51 To start the file cleaner immediately, click on the icon [[image:1700040675471-344.png]].
52
53 This activation of the maintenance service has a specific behaviour equivalent to the //FORCE_FILESMAINTENANCE //internal event //: //the file cleaner is asked to start a cleaning operation. All current sessions are closed. The session manager and scheduler are stopped and restarted once the operation is complete.
54
55 == From the digdash.properties file ==
56
57 File modified: **digdash.properties**
58 See chapter "[[Externalizing parameters to a properties file>>doc:Digdash.deployment.configuration.advanced_system_guide.other_advanced_parameters.WebHome||anchor="externalisation"]]"to perform this operation.
59
60 Activate or not the Files GC module and/or launch the cleanup at server startup.
61
62 (% class="box infomessage" %)
63 (((
64 ℹ Tasks in progress (scheduled tasks) will cancel the maintenance service. A new attempt will be made 1 hour later. After 5 failed attempts, the service will no longer pass (5 new attempts maximum) and will be postponed until the next scheduled maintenance date.
65 )))
66
67 Available parameters :
68
69 * //Name//: **ddenterpriseapi.startCleaner**
70 //Value//: Boolean (default: false)
71 //Description//:
72 ** true: automatic file cleanup scheduled.
73 //Note: the cleanup time is defined in **system.xml**, by the **FILESGC_SCHEDXML** parameter.//
74 The//default cleanup time (if none is specified in system.xml, FILESGC_SCHEDXML) is every day at 0:00//
75 ** false (default): file cleaner not used
76
77 * //Name//: **ddenterpriseapi.cleanOnStart**
78 //Value//: Boolean (default: false)
79 //Description//:
80 ** true: cleans up unused files at server startup (history files, cubes, results files, etc.)
81 ** false (default): does not clean up unused files at server startup
82
83 * //Name//: **ddenterpriseapi.autoBackup**
84 //Value//: Boolean (default: false)
85 //Description//:
86 ** ***. true: enables scheduled automatic backup.**
87 ** false (default): does not enable scheduled automatic backup
88
89 (% class="box warningmessage" %)
90 (((
91 In the **digdash.properties** file, all parameters are prefixed with the name of the application concerned.
92 Here, the default is ddenterpriseapi (ddenterpriseapi.war).
93 If you have renamed this application, for example to **ddapi_dev.war**, the parameters become **ddapi_dev.startCleaner**, **ddapi_dev.cleanOnStart, ...**
94 )))
95
96 = Programming and options for automatic maintenance{{id name="Options_auto"/}} =
97
98 File modified: **system.xml**.
99
100 The available parameters are listed below.
101
102 (% class="wikigeneratedid" id="HFILESGC_SCHEDXML" %)
103 **FILESGC_SCHEDXML**
104
105 //Value//: XML sentence (encoded) (default: none)
106 //Description//: This parameter contains an //encoded// XML sentence describing the scheduling frequency.
107
108 Example:
109
110 {{code cssClass="notranslate" language="xml"}}
111 <Property key="FILESGC_SCHEDXML"
112 value="&lt;Schedule frequency=&quot;daily&quot;
113 fromDay=&quot;11&quot; fromHour=&quot;0&quot;
114 fromMinute=&quot;0&quot; fromMonth=&quot;7&quot;
115 fromYear=&quot;2021&quot; periods=&quot;1&quot;
116 time=&quot;0:0&quot;/&gt;"></Property>
117 {{/code}}
118
119 The attributes are as follows:
120
121 * **frequency **: **hourly**, **daily, weekly **or **monthly**.
122 * **fromDay**, **fromHour**, **fromMinute, fromMonth**, **fromYear**: start date (//❗fromMonth// starts at 0 for January). In general, you should enter a date in the past.
123 * **periods **:** number of **hours, days or months (depending on the selected frequency) between 2 cleanings.
124 * **time** (cleaning time for the daily, weekly and monthly frequencies.
125
126 The example above means every day (frequency="daily" and periods="1") at 0:00 (time="0:0").
127
128 For a weekly frequency (weekly), we will have an additional **weekDays** attribute with the following values (to be added together for a trigger on several days of the week. For example, Monday and Wednesday: 2 + 8 = 10):
129
130 * Sunday: 1
131 * Monday: 2
132 * Tuesday: 4
133 * Wednesday: 8
134 * Thursday: 16
135 * Friday: 32
136 * Saturday: 64
137
138 For example, for a schedule every 2 weeks, on Mondays and Thursdays, at 18:00, from 1 January 2025 00:00, the property will have the following value:
139
140 {{code language="xml"}}
141 <Property key="FILESGC_SCHEDXML"
142 value="&lt;Schedule frequency=&quot;weekly&quot;
143 fromDay=&quot;1&quot; fromHour=&quot;0&quot;
144 fromMinute=&quot;0&quot; fromMonth=&quot;0&quot;
145 fromYear=&quot;2025&quot; periods=&quot;2&quot;
146 time=&quot;0:0&quot; weekDays=&quot;18&quot;/&gt;">
147 </Property>
148 {{/code}}
149
150 **FILESGC_SESSIONSCHECK**
151
152 //Value//: true/false (Boolean) (default: none, equivalent to true)
153 //Description//: This parameter indicates whether the file cleaner should check for active sessions (connected users) before launching (true), or whether it launches regardless of the status of active sessions (false). In the latter case, all sessions will be disconnected instantly.
154
155 Example:
156
157 {{code language="xml" cssClass="notranslate"}}
158 <Property key="FILESGC_SESSIONSCHECK" value="false"></Property>
159 {{/code}}
160
161 **FILESGC_TASKSCHECK{{id name="task"/}}**
162
163 //Value//: true/false (Boolean) (default: none, equivalent to true)
164 //Description//: This parameter indicates whether the file cleaner should check active tasks before starting (true), or whether it should start regardless of the status of active tasks (false). In the latter case, all tasks will be aborted.
165
166 Example:
167
168 {{code language="xml" cssClass="notranslate"}}
169 <Property key="FILESGC_SESSIONSCHECK" value="false"></Property>
170 {{/code}}
171
172 (% class="box" %)
173 (((
174 💡 **Combination of //FILESGC_SESSIONSCHECK// and //FILESGC_TASKSCHECK// parameters**
175
176 * __Case 1 __: //FILESGC_SESSIONSCHECK = true// and //FILESGC_TASKCHECK = true// (default behaviour).
177 ➡ The maintenance service only passes if no user session or task is active.
178 * __Case 2 __: //FILESGC_SESSIONCHECK = false// and //FILESGC_TASKCHECK = true//.
179 ➡ The maintenance service runs regardless of the status of the sessions but is cancelled if any tasks are active.
180 * __Case 3 __: //FILESGC_SESSIONCHECK = false// and //FILESGC_TASKCHECK = false//.
181 ➡ The service runs regardless of the status of the sessions and tasks. Active sessions are disconnected and active tasks are stopped.
182 * __Case 4 __: //FILESGC_SESSIONCHECK = true// and //FILESGC_TASKCHECK = false//.
183 ➡ The maintenance service passes if there are no active sessions (user connected), whatever the status of the tasks. If this is the case, the tasks are interrupted.
184 )))
185
186 **USEAUTOBACKUP{{id name="auto"/}}**
187
188 //Value//: true/false (Boolean) (default: none, equivalent to false)
189 //Description//: This parameter indicates whether the maintenance service also performs a full configuration backup before executing the file cleanup.
190
191 (% class="box" %)
192 (((
193 💡 **Scheduling automatic backup**
194 \\Automatic backup is performed according to the schedule defined for automatic maintenance (parameter //FILESGC_SCHEDXML)// regardless of the status of sessions or tasks. It is therefore possible for automatic backup to be carried out but for file cleanup not to be performed. For example, if the maintenance service checks the active tasks before starting and there are any.
195 The automatic backup will not be carried out when new attempts are made to clean up the files.
196 )))
197
198 **PROP_FILESGC_MIN_FILE_AGE**
199
200 //Value//: number (minutes) (default: 0)
201 //Description//: This parameter specifies the minimum age in minutes that a file must be to be deleted (if it is to be deleted). If the file is newer than this value then it will not be deleted.
202
203 **PROP_FILESGC_FASTMODE**
204
205 //Value//: true/false (Boolean) (default: none, equivalent to false)
206 //Description//: This parameter indicates whether the maintenance service must open a session for each system user (false, the normal case) or whether it can do without (true). If it is enabled, then the scan of files to be cleaned is faster, but there is a risk that some files will not be deleted, for example those of flows and models that have been deleted but still have data on disk.
207
208 **PROP_FILESGC_MAXTIME**
209
210 //Value//: number (minutes) (default: -1, equivalent to no limit)
211 //Description//: This parameter specifies the maximum time dedicated to cleaning files. If the time is exceeded, file cleaning is interrupted and there will be files left to clean the next time. //Warning: if this time is unreasonably short, the cleaner may do nothing. This value is highly dependent on the Digdash configuration and should be set according to the average cleanup times observed.//
212
213 **PROP_FILESGC_RETRY_MAX_TIME**
214
215 //{{id name="retry"/}}Value//: number (minutes) (default: 360)//
216 Description//: This parameter specifies the maximum length of time that the file cleaner will retry cleanup attempts in the event that the cleanup has failed (due to active sessions or tasks).
217 For example, if the value is set to 360, it will retry for a maximum of 6 hours after the first attempt.
218
219 **PROP_FILESGC_RETRY_PERIOD**
220
221 //Value//: number (minutes) (default: 60)//
222 Description//: This parameter specifies the time between 2 file clean-up attempts if the clean-up has failed (due to active sessions or tasks).
223 For example, if the value is set to 60, it will retry every hour if the previous attempt failed, for the maximum time defined in the //PROP_FILESGC_RETRY_MAX_TIME// parameter.