Adding a JDBC driver

Last modified by Aurelie Bertrand on 2022/04/05 12:10

This document describes the process to add a new JDBC data base driver to DigDash Enterprise.

Requirement

The JDBC driver as a JAR file, and its documentation

Installation

Driver files deployment

  1. (optional, if the webapps have not been already deployed). Start your DigDash Enterprise Server and wait for the webapps to be fully deployed
  2. Stop the server
  3. copy the driver's JAR file into the following folders:
    <DD Install>/apache-tomcat/webapps/ddenterpriseapi/WEB-INF/lib
    <DD Install>/apache-tomcat/webapps/studio/WEB-INF/lib
  4. The JDBC driver must be registered within DigDash Enterprise Server.

Registering the driver in Digdash Enterprise

Edit the following file :

<DD Install>\apache-tomcat\webapps\ddenterpriseapi\WEB-INF\classes\resources\config\sqldriverrepository.xml

Add an XML entry to the sqldriverrepository.xml that looks like the following sample:

<SQLDriver id="MY_DRIVER"
          name="My Driver"
          url="mydriver:"
          manufacturer="My Driver Company"
          class="com.mydriver.MyDriver"
          urlsample="jdbc:mydriver:<database>?<options>"
          availability="both">
   <properties></properties>
</SQLDriver>

Important: Some characters are reserved in XML: '&', '"', '<' or '>'. If you need these characters you must encode them into their corresponding XML entity.

Corresponding XML entities:

  • & => &amp;
  • " => &quot;
  • < => &lt;
  • > => &gt;

Example:

  • (WRONG) urlsample="jdbc:mydriver:dbtest?opt1=0&opt2="value""
  • (RIGHT) urlsample="jdbc:mydriver:dbtest?opt1=0&amp;opt2=&quot;value&quot;"

Parameters:

  • id: An identifier used by DigDash internally, choose any non existent string, the convention is uppercase, no spaces
  • name: The driver's name to be displayed in the user interface of the administration console
  • url: jdbc URL prefix (without "jdbc:"). See driver's documentation
  • manufacturer: Name of the driver's vendor/developer
  • class: java class path of the main driver class. See driver's documentation.
     Optional: JDBC drivers compliant with JDBC 4 do not need a driver class. Keep the class attribute empty (class="") in that case.
  • urlsample: User friendly URL sample for the user interface of the administration console.
  • availability: reserved. Leave it at "both".

Important: The deployment is lost when upgrading ddenterpriseapi.war file.
Please follow the deployment guide each time you upgrade DigDash Enterprise

You can find some sample of XML for drivers not provided in DigDash enterprise in sqldriverrepository.xml file.

Specific properties (advanced)

You can specify secific properties on some JDBC drivers. These properties are written inside the <properties></properties> tag, under the following format:

<properties>

<property name="property_name" value="property_value" ></property>

<properties>

Supported properties are:

FORCE_FORWARD_ONLY (undefined | false | true)

Description: specifies the type of JDBC result set cursor used by the Studio for the preview of the SQL query results. By default the Studio uses a TYPE_INSENTIVE_SCROLL cursor to preview the results, but some of the databases do not support this type of cursor. If your driver or database only supports TYPE_FORWARD_ONLY cursors you can specify it with FORCE_FORWARD_ONLY property. Possible values:

  • false (or undefined property): The type of cursor is automatic, TYPE_SCROLL_INSENTIVE in most of the cases, except for HIVE, IMPALA and SAPHANA
  • true: The type of cursor used in the Studio is TYPE_FORWARD_ONLY

PING_SQL (undefined | SQL query | empty string)

Description: DigDash Enterprise tests the connection with the database by using the JDBC method Connection.isValid(). On some drivers this method does not work. In that case DigDash uses an alternative method to “ping” the database, usually by sending a “select 1” statement.

The PING_SQL property allows you to specify this SQL statement depending on your driver or database. Possible values:

  • Undefined property: The alternative ping query is automatically decided by DigDash: “select 1” except for ORACLE, FIREBIRD, SAPHANA, DB2_AS400 or DB2 drivers
  • SQL query (not empty): the specified query is used to ping the databse. Example:
     <property name="PING_SQL" value="select 1 from all_tables" />
  • Empty string: Special case used to deactivate the ping in the case the JDBC method Connection.isValid() fails. The database is considered to be always accessible. Example:
     <property name="PING_SQL" value="" />

USE_FETCH_FIRST_IN_STUDIO (undefined | false | true)

Description: This property is used only for preview the SQL result in the data source configuration dialog (Studio). It modifies the query by adding “FETCH FIRST n ROWS ONLY” to it (n is replaced byt the number of preview rows). It is useful for drivers that do not support JDBC’s Statement.setMaxRowsthumb_down, for example AS400 JDBC driver. Possible values:

  • false (or undefined property): The preview limit is defined by using JDBC method Statement.setMaxRowsthumb_down
  • true: Preview limit is defined by adding FERTCH FIRST n ROWS ONLY to the SQL in the Studio.

FORBID_POOL_CONNECTION (undefined | false | true)

Description: Prevents DigDash Enterprise from using a JDBC connection pool for this driver. A JDBC connection pool optimizes access to the data base by keeping connections in an opened state, and reuse them for different queries. In some cases it is preferred to not use a JDBC connection pool, for example to ensure that connections will not stay opened too long on the data base after they have been used for a query. This property answers that need. Possible values:

  • false (or undefined property): A JDBC connection pool may be used by the driver
  • true: The JDBC connection pool will not be used by the driver and each SQL query will have its own connection.

POOL_VALIDATION_QUERY (undefined | SQL query)

Description: Specify a validation query used by the JDBC Apache DBCP2 connection pool. Some drivers do not specify this query (eg. « select 1 ») and can malfunction when a connection is created by the JDBC connection pool. This property is similar to the validationQuery property which is configured in Apache DBCP2 pool configuration. Possible values:

  • undefined: No specified validation query for this driver. The connection pool will use the default validation query, if it exists.
  • non-empty SQL query : The specified query will be use to test the validity of a connection obtained from the Apache DBCP connection pool. (Example : select 1)

DEFAULT_FETCH_SIZE (undefined | number)

Description: Specify a maximum number of rows to be retrieve at once by DigDash Enterprise. Some JDBC drivers (eg. Postgresql) retrieve all result rows of a request at once, instead of streaming the result to be consumed by DigDash Enterprise. This can lead to a higher memory usage (bigger memory buffer needed) and to prevent cancelling the fetch of the result mid-time. This parameter can help avoiding this case by specifying a reasonable number of rows to retrieve (eg. 100). For example, DEFAULT_FETCH_SIZE = 1000 will retrieve the first 1000 rows of result, and when DigDash will need the 1001st row, the driver will retrieve the next 1000 rows, and so on. This parameter can also by specified directly in the data source configuration (advanced) in the Studio.

Pre-configured drivers, not included

The following Database connectivities are available in DigDash Enterprise by default, but their JDBC driver is not included and need additional installation:

MySQL

DigDash Enterprise can connect to a MySQL 5.5 (and after) database by using MariaDB Java connector driver. In most of the cases, this driver is enough to connect to a MySQL (and MariaDB) database. However, to connect to a previous version of MySQL, or to use specific features of the native MySQL JDBC driver, you will need the MySQL driver corresponding to your version of the database. For these cases, you must download the driver from MySQL website: https://dev.mysql.com/downloads/connector/j/

The MariaDB driver included in DigDash Enterprise handles jdbc:mysql:// et jdbc:mariadb:// connections. To use the MySQL native JDBC driver you must remove MariaDB driver from the ddenterpriseapi and studio webapps. The file to remove is mariadb-java-client-*.jar. Keeping both drivers can lead to issues when connecting to MySQL.
However, it is possible to keep the MariaDB driver, for the case when you have a MySQL database and a MariaDB database. You need to add the parameter &disableMariaDbDriver in the connection string to your MySQL database (jdbc:mysql://...&disableMariaDbDriver). More information on the MariaDB website: https://mariadb.com/kb/en/about-mariadb-connector-j/.

If you decide to install the MySQL driver, you can notice that a definition for it already exist in the file sqldriverrepository.xml included in DigDash Enterprise (see XML below), but it is commented out. You need to un-comment this XML definition (and also comment the one for MariaDB driver).
This definition concerns the newer version of the MySQL driver, from MySQL Connector/J 8.0. Recent MySQL drivers are compatible with all version of MySQL database. However, if you decide to install an older version you need to remove the class property from this definition, and then rename the classOld property into class, because the driver's class name has changed in Connector/J 8.0. In any case, read the documentation corresponding to the driver you install.

 <SQLDriver id="MYSQL" name="MySQL"
   url="mysql://"
   class="com.mysql.cj.jdbc.Driver"
   classOld="com.mysql.jdbc.Driver"
   manufacturer="MySQL"
   urlsample="$ui.DataBase.MySqlSampleUrl"
   availability="both">
 <properties>
 </properties>
</SQLDriver>

Teradata

The JDBC driver for Teradata can be downloaded on Teradata website: https://downloads.teradata.com/download/connectivity/jdbc-driver. The download needs a user account.

IBM DB2

The JDBC driver for IBM DB2 can be downloaded on IBM website: https://www.ibm.com/support/pages/db2-jdbc-driver-versions-and-downloads. The download needs a user account. The driver consist in 2 .jar files, one for the driver, one for the license. You can also find these files in your DB2 installation.

Microsoft SQL Server

DigDash Enterprise is packaged with jTDS, an open source JDBC driver compatible with Microsoft SQL Server. However, if you want to use specific features of the Microsoft SQL Server driver, you need to download it from Microsoft website: https://docs.microsoft.com/fr-fr/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server?view=sql-server-ver15. 

Hadoop Hive

Installing Hadoop Hive driver depends on your Hadoop installation (Example: Cloudera, Hortonworks...). It it recommended to use the corresponding driver included with your Hadoop Installation (non exhaustively) :

Google BigQuery

Read the specific document about BigQuery Connector in Digdash Enterprise.

SAP HANA

Read the specific document about SAP HANA in Digdash Enterprise.