Advanced performance settings

Last modified by Aurelie Bertrand on 2025/12/19 08:51

Modified file: system.xml

Example of XML syntax :

<Property key="CORE_TP_EXECSIZE" value="64"></Property>

Threads used to execute programmed Flows

Modifies the number of threads used to execute programmed (scheduler) or event-driven Flows.

Available parameters:

  • Name: MAX_TP_EXECSIZE
    Value : integer > 0 (default: 16)
    Description: Maximum number of threads processing synchronisation tasks.
     
  • Name: TP_SYNC_PRIORITY
    Value : string ("flow" or "none") (default: flow)
    Description: Task priority mode.
    Value set to flow : Processes the synchronisation of the flow as soon as possible after the cube has been generated for this flow.
    Value set to none : the flow will be synchronised when there is room in the thread queue. This parameter is only taken into account when the TP_PRIORITYPOOL parameter is set to true.
     
  • Name: TP_SYNC_GROUPFLOWBYCUBE
    Value : Boolean (default: false)
    Description: Changes the processing mode for pending tasks.
    Value set to false : Flow processing tasks are distributed across all available threads, regardless of the cube used. This results in parallel processing of flows to the detriment of cubes, which is recommended when there are few cubes but many flows.
    Value set to true : groups the processing of Flows from the same cube into a single thread. This results in parallel processing of cubes to the detriment of flows, recommended when there are many different cubes and few flows using the same cubes. This parameter is only taken into account when the TP_PRIORITYPOOL parameter is set to true.

Threads used to execute interactive flows

Modifies the number of "threads" used for interactive Flow execution (Studio, dashboard, mobile, etc).

Available parameters:

  • Name: MAX_TP_PLAYSIZE
    Value : integer > 0 (default: 16)
    Description: Maximum number of threads for processing synchronisation tasks.
     
  • Name: TP_PLAY_PRIORITY
    Value : string ("flow" or "none") (default: flow)
    Description: Task priority mode.
    Value set to flow : Processes the synchronisation of the flow as soon as possible after the cube has been generated for this flow.
    Value set to none : the flow will be synchronised when there is room in the thread queue. This parameter is only taken into account when the TP_PRIORITYPOOL parameter is set to true.
     
  • Name: TP_PLAY_GROUPFLOWBYCUBE
    Value : Boolean (default: false)
    Description: Changes the processing mode for pending tasks.
    Value set to false : Flow processing tasks are distributed across all available threads, regardless of the cube used. This leads to parallel processing of flows to the detriment of cubes, which is recommended when there are few cubes but many flows.
    Value set to true: groups the processing of flows from the same cube on a single thread. This results in parallel processing of cubes to the detriment of flows, recommended when there are many different cubes and few flows using the same cubes. This parameter is only taken into account when the TP_PRIORITYPOOL parameter is set to true.

Delays in deleting cubes from memory

Modifies the way in which the Cube Manager deletes unused cubes from memory.

The following settings change how cubes that have not been used for a certain amount of time are deleted, even if the session is still active.

Available parameters:

  • Name: CUBE_TIMEOUT_INTERACTIVE
    Value: minutes > 0 (default: 10 minutes)
    Description: Duration of the inactivity period for a cube loaded in interactive mode (cube browsing on the server).
     
  • Name: CUBE_TIMEOUT_SYNC
    Value: minutes > 0 (default: 4 minutes)
    Description: Duration of the period of inactivity for a cube loaded in programmed mode (generation of a programmed flow).
     
  • Name : CUBE_TIMEOUT_PERIOD
    Value: minutes > 0 (default: 2 minutes)
    Description: Cube inactivity check interval, should be at least CUBE_TIMEOUT_SYNC / 2

Data cube performance

These parameters will affect the interactive processing of data cubes (flattening into result cubes during display). These parameters do not affect the generation of data cubes.

Available parameters:

  • Name: CUBEPART_MAXSIZEMB
    Value: megabytes > 0 (default: 100 MB)
    Description: Size of a cube part in megabytes. A cube share is a part of the data cube that can be processed (flattened) in parallel or distributed to other DigDash Enterprise servers in cluster mode (see the"Using multiple servers in cluster mode" chapter in this document).
     
  • Name: TP_MCUBESIZE
    Value: threads > 0 (default: 64 threads)
    Description: Size of the thread queue used for parallel processing of cube shares. Large cubes (e.g. several million/thousand lines) are processed in parallel by the server and/or by other servers (in cluster mode). This parameter is the number of parallel processing units (threads) on a machine. Each cube share occupies one unit of the queue while it is being processed; if the queue is full, additional units are put on hold.
     
  • Name: MCUBE_ROWS_PER_THREAD
    Value: rows > 0 (default: 100000)
    Description: Limit on the number of rows in a data cube, beyond which the DigDash Enterprise server will activate parallel processing of the cube shares (if there is more than one share for this cube). Below this limit, cube processing is sequential rather than parallel.

Data cube size

The following parameters are used to limit the size of cubes when they are generated or loaded.

Available parameters:

  • Name: RESULTCUBE_MAXSIZE / RESULTCUBEPART_MAXSIZE
    Value: rows > 0 (default: -1, disabled)
    Description: Maximum number of rows in the intermediate cube. This prevents the generation of a cube that is too large for the memory. If the limit is exceeded, an error message is displayed (Too many rows).
    ❗ These parameters impose a limit on the generation of the results cube. So, even if you perform an operation such as a "Top 3", it is possible to exceed this limit (and cause an exception), because the entire dataset is built to extract this "Top 3".
     
  • Name: PROP_RESULTCUBE_MAXSIZE_FOR_BROWSER 
    Value: values > 0 (default: -1, disabled)
    Description: Maximum number of values contained in the result cube. This is used to protect the browser from downloading a large result cube (JSON). The result cube is generated but if it contains more values than the defined limit, JSON generation fails and an error message is displayed.

💡 The number of values is not the same as the number of rows in an array. There is usually one value per measure and per axis intersection. For example, if you have a table with 1,000 rows and 4 measurements, the result cube can contain up to 4,000 values (masked measurements and objective measurements are also counted). Empty (non-existent) values are not counted. Therefore, if your table has a lot of empty values, you risk having fewer values for the same number of rows.

  • Name: PROP_FREEMEM_MINRATIO_LOAD_CUBE
    Value: decimal number (default: -1, disabled)
    Description: Free memory must be greater than this number multiplied by the size of the cube for the cube to be loaded.

Other performance parameters

The following parameters are used to analyse or optimise system performance.

Available parameters:

  • Name: LOW_MEMORY_THRESHOLD
    Value: percentage > 0 (default: 10%)
    Description: Threshold in percentage of free memory remaining below which the system will issue a low memory alert. This alert is visible on the server status page for 24 hours. It is also recorded in the DDAudit database if the system audit service is started.
    Finally, a DigDash event is also triggered when the threshold is reached: SYSCHECK_LOWMEM. An example of the use of this event is given in the DDAudit module deployment document.
     
  • Name: PROP_CUBE_GEN_OOM_PREDICTION_THRESHOLD 
    Value: percentage > 0 (default: 10%)
    Description: Threshold in percentage of free memory remaining below which the system may decide to interrupt the generation of a cube, in order to avoid an insufficient memory situation (OOM: Out Of Memory).

    For example, if : PROP_CUBE_GEN_OOM_PREDICTION_THRESHOLD = 10
    then if the free memory is less than 10% of the total memory for at least one minute, the generation of a cube in progress at that time is interrupted (failure) in an attempt to avoid a situation of insufficient memory which could be fatal to the server (a restart may be necessary). If this is not enough to bring the free memory above the threshold, the generation of another cube is interrupted, and so on.
     

  • Name : PROP_CUBE_GEN_OOM_RETRY_COUNT 
    Value: integer (default: 1)
    Description: Number of cube generation attempts following an interruption in cube generation due to insufficient memory.
    When the server runs out of resources to generate new tasks, the generation of a cube may be interrupted in order to avoid an insufficient memory situation (property above). In this case, the generation of the cube is automatically rescheduled at the end of the batch for the number of times defined here.
    This rescheduling is visible in the server logs in the following form:
    2025-08-07 11:06:05,260 DEBUG [ddenterpriseapi] [SyncTP-4-thread-15] (DataSourceUnit.java:108) - com.digdash.exceptions.ExceptionCantBuildCube: Too much memory used => retry ExecutionContext(ItemIdentifier(admin|7e7b185b|-1)|#15)
     
  • Name: TP_PRIORITYPOOL
    Value: Boolean (default: true)
    Description: Uses a pool of refresh threads with priority management for Flow and cube refreshes. See parameters TP_PLAY_GROUPFLOWBYCUBE, TP_SYNC_GROUPFLOWBYCUBE, TP_PLAY_GROUPFLOWBYCUBE, TP_SYNC_GROUPFLOWBYCUBE
     
  • Name: PROP_JS_OPTIM_LEVEL
    Value: optimisation level from -1 to 9 (default: -1)
    Description: Used to define the optimisation level used when compiling javascript. -1 disables compilation, 0 disables optimisation only. The maximum optimisation level is 9, although it is possible that optimisation may cause errors during compilation.
     
  • Name: PROP_BUILDER_MTHREAD_POOLSIZE
    Value: integer (if not specified, the value is the number of threads on the machine)
    Description : Used to define the maximum number of threads allocated for generating an HTML Builder. The value chosen depends on the number of threads on the machine and what you want to do. The use of several threads allows the different HTML Builder flows to be generated in parallel in order to optimise generation time. However, it is necessary to limit the number of threads in order to avoid an overall slowdown.