Module SOAP

Last modified by jhurst on 2021/04/21 10:01

SOAP module - Documentation

Retreiving a data source via a SOAP web service

Summary :

Prerequisite

  • Acronyms in this document are defined in the lexicon at the end.
  • Make sure the folder <digdash_installation>/add-ons/soap is provided in your DigDash installation. It contains the web application needed to make SOAP calls from DigDash. The installation procedure is described in this document.
  • Make sure you have all the needed WSDL files describing all the services to call
  • Make sure you know at least the SOAP requests structure (in XML), the methods to call as well. A documentation of the Service Provider (that DigDash does not provide) may be necessary.
  • Make sure you know the different data formats that you will need to mention. A documentation of the Service Provider (that DigDash does not provide) may be necessary.
soap_en_html_c801e156356d3a0a.png
 

DigDash offers a helper to preview a SOAP request.

Go to

<digdash_server>/ddsoap?about=true

to have information about your SOAP configuration.

Description

General

DigDash Enterprise offers the possibility to retrieve data sources calling SOAP web services.

SOAP communication is possible via the XML format. The point is to get the SOAP reponses in XML format and to use them directly as a data source as DigDash already knows how to proceed this format.

To do so, DigDash will mainly work with the web services’ interfaces, known as WSDL files that you must provide. Indeed, these files provide a description of what the web service to call offers, in detail.

This add-on is based on the SOAP UI API (SOAP UI is a well known SOAP client). The way you call the web service will be kind of the same way SOAP UI does it. Namely you will have to fill in the values of the SOAP request parameters.

Hence the importance of knowing the SOAP request structure in XML as said in the prerequisite.

Technical description

DigDash Enterprise offers this functionality as a module (or add-on) to install (it is not natively installed). We will use the existing principle of WLNK (Web Link) files in DigDash to host the information needed to retrieve a data source via SOAP.

A WLNK (Web Link) file in DigDash contains a URL. The content of this URL is downloaded by the DigDash server every time is needed (for a data source or on the user demand).

Why create a WLNK file?

We could create a simple XML file as a data source but this one would be just a snapshot of the data. The WLNK file allows you to refresh the data every time when needed.
Be careful, a WLNK file only contains the information for one and only one SOAP request.
1 WLNK file = 1 SOAP request

DigDash will never connect to the web service endpoint directly. Indeed, DigDash will call a web application instead (deployable on the same DigDash Tomcat server) developed by DigDash. The add-on mainly consists of this web application.

The goal of this web application is to reach the web service with the information you will have provided. It eventually will return the SOAP response to DigDash in XML.

soap_en_html_5e3e099e5561dd5e.png
 

Schema : Description of SOAP calls from DigDash

Installation and configuration of the SOAP module

Installation and deployment of the web application

First of all, stop the Tomcat server.

Copy the web application ddsoap.war located in <digdash_installation>/add-ons/soap to <digdash_install>/apache-tomcat/webapps.

Restart the DigDash server to deploy the web application.

wsdls folder

Place in the folder : « <digdash_installation>/add-ons/ddsoap/wsdls » the list of the WSDL files (.wsdl) describing the web services you want to call.

soap_props.properties configuration

Configure the ddsoap/soap_props.properties file with the following properties:

Miscellaneous properties

PropriétésDescription
PRINT_DEBUG

This property must be set to ‘true’ to display any traces. Only error traces are always displayed.

This property allows to display debug traces in logs.

SHOW_REQUESTSAllows to display the SOAP requests (in XML) sent to the Web Service.
SHOW_RESPONSESAllows to display the SOAP responses (in XML) sent by the Web Service.
LOGIN_BEFORE_CALL_SERVICEAllows authentication before any SOAP call. Credentials properties must start with _creds (see next point).
REMOVE_REQUEST_EMPTY_VALUESRemoves empty values (identified by « ? ») of the SOAP request.
REMOVE_REQUEST_EMPTY_NODESRemoves empty nodes of the SOAP request.

Table : Miscellaneous properties for the SOAP module configuration

Credentials properties

Authentication may be necessary before any SOAP calls.

ALL these properties MUST start with « creds_ » AND the property LOGIN_BEFORE_CALL_SERVICE must be set to ‘true’ 

PropertiesDescription
Caution : You must escape special characters in keys (property name) if needed (use \= for = , use \: for
creds_binding(Mandatory) The name of the authentication service.
creds_operation(Mandatory) The name of the authentication method.
creds_<username>The possible login name for authentication.
creds_<password>The possible password for authentication.
creds_<any other parameter>The possible other authentication parameters.

Table : Properties used for authentication for the SOAP module

Configuration in DigDash Enterprise

Creation of a WLNK file

Let’s start with extracting data from an XML file.

Data source > Add > Any file…

Select the documents server to use

Add a new document to the server clicking the « Add a file to the server... » button.

Three options are available. To get the content via SOAP, we are going to select the second option « Enter a URL ».

→ The URL to call is the URL of the web application, ddsoap previously deployed (basically <digdash_server>/ddsoap).

→ The name of your WLNK file must end with .xml

→ Check the option « Only add the URL link to the Documents server ». This option will create the WLNK file.

→ It is possible to customize the request timeout (in seconds) for a WLNK (only values > 0 will be taken into accout, the default request timeout will be applied otherwise).

soap_en_html_ce75b9c82fc22c3.png

 

Screenshot: creation of the WLNK file

You can enter the SOAP parameters as key-value in the GET parameters.

You IMPERATIVELY must at least mention these two parameters:

_binding : the name of the Service to call
_operation : the name of the operation (or method) of the Service to call

soap_en_html_1da100bad7bbced9.png

 

Screenshot: SOAP parameters

For the other parameters, you will need to mention their name (do not forget the namespace). If the name appears more than once in the request, you will need to specify its parents to access this specific node.

In general, the first node in the request with the specified name will be selected unless the path to a specific node is specified (without needing to start from the root node).

SeparatorsDescriptionExample
/Path separator in the XML.

<a>

<a1>

<b>?</b>

<c>?</c>

</a1>

<b>?</b>

<c>?</c>

</a>

<a1>

<b>?</b>

<c>?</c>

</a1>

To access the element c, we will specify a/c (specifying c will select the first element c)

@To access an element attribute.

<a>

<b>?</b>

<c attr=?>?</c>

</a>

To access the attribute attr of the element c, we will specify a/c@attr (or c@attr directly)

Table:Naming rules to access a specific node

Once the SOAP parameters are entered, click on « OK » and the WLNK file will be created and uploaded to the selected documents server.

This WLNK file can now be loaded as a classical XML data source file.

The created WLNK file can be modified later if needed. Right click on your WLNK file and open the menu "Configure the WLNK file" to edit the link and the request parameters.

1617957700408-612.png

1617957937570-960.png

Pagination management

The SOAP module can help you gather and get all data in DigDash from a request with pagination.

You must provide the following GET parameters to your URL :

_pagination : this parameter must be set to true to specify pagination is required.

_pageTag : the value must be the name tag of the element or attribute that drives the pagination (see table : Naming rules to access a specific node to define this parameter).

_pageStartElement : you must provide the name of the structure containing the lines we are interested in (see next part). If you do not know the SOAP response structure, you might have to send a first request first without pagination to know this tag name (see table : Naming rules to access a specific node to define this parameter).

_pageInc : the index of the page starts with 0. this parameter must define the incrementation of this index.

_resultSize (optional) : specifying the result size can optimize the response time.

IV.3Configuration of the XML data source

When loading the XML response into DigDash, the data source may be difficult to read at first glance.

Indeed, you need to mention where the data source must start the lecture in the XML structure. This is possible in the advanced options of the data source.

For example : let’s consider the following SOAP response in XML :

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

<soapenv:Body>

<a></a>

<structs>

<struct1>

<s11></s11>

<s12>

<s121></s121>

</s12>

<s13></s13>

<s14></s14>

</struct1>

<struct2>

<s21></s21>

<s22>

<s221></s221>

</s22>

<s23></s23>

<s24></s24>

</struct2>

<struct3>

<s31></s31>

<s32>

<s321></s321>

</s32>

<s33></s33>

<s34></s34>

</struct3>

</structs>

</soapenv:Body>

</soapenv:Envelope>

 

Example : Example of a SOAP reponse in XML

We need to identify the structure containing the structures (the lines) we are interested in. In this example, the structures struct1, struct2 and struct3 are our lines.

The structure structs is the structure that contains our lines.

So we need to specify it in the advanced options of the data source ;

soap_en_html_1934d3df1390c07f.png

 

soap_en_html_7f1972291bc6cc7b.png

 

Check the option « Start reading XML from a specified tag » and mention the strucure we want to start reading from.

Also, we will uncheck the option « Expand child nodes and attributes into columns ».

Lexicon


    • In this document, we give the following definitions :

      SOAP : Simple Object Access Protocol. It is a messaging protocol specification for exchanging structured information in the implementation of web services in computer networks. It uses XML Information Set for its message format […]. (Wikipédia)

      WSDL : Web Services Description Language. It is an XML file describing a web service.

WLNK : Web LiNK. Files with the extension .wlnk host links to a data source.

References

DigDash uses the Open Source API SOAPUI from SmartBear Software to support SOAP communication with web services

https://www.soapui.org