Wiki source code of Module SOAP

Last modified by jhurst on 2020/07/21 11:35
Show last authors
1 {{ddtoc/}}
2
3 ----
4
5 This document describes how to connect to a data source by querying a SOAP web service.
6
7 = Prerequisites =
8
9 * The acronyms used below are referenced in the [[lexicon>>doc:||anchor="Lexique"]] at the end of this document.
10 * Have the **<digdash_installation>/add-ons/soap** folder containing the web application needed to set up calls to SOAP Web Services from DigDash. The installation of this add-on will be described in this document.
11 * Have a list of WSDL files corresponding to the Services we want to call.
12 * Have a basic understanding of the structure of SOAP requests (in XML) and the methods to be called. Documentation from the Service Provider (which DigDash does not supply) may be required.
13 * Knowledge of the formats of the SOAP request parameter values to be filled in, documentation from the Service Provider (which DigDash does not provide) may be required.
14
15 (% class="box infomessage" %)
16 (((
17 💡 DigDash provides help for consulting the structure of SOAP requests.
18
19 Go to **<digdash_server>/ddsoap?about=true** for information about your SOAP configuration.
20 )))
21
22 = Description =
23
24 == General ==
25
26 DigDash offers the ability to retrieve data sources by querying SOAP web services.
27
28 Since SOAP exchanges take place via messages in XML format, the aim is to be able to retrieve SOAP responses in XML format from the web service and use them directly as XML data sources that DigDash can process.
29
30 To do this, DigDash will work mainly with the web service interfaces, the WSDL files that you will need to supply. These provide a detailed description of what the web services to be called offer.
31
32 DigDash is based on the Java API of the famous SOAP client, SOAP UI. The logic and the way of querying the desired web service will be quite similar, in the sense that you will have to fill in the values of the request parameters.
33
34 Hence, as indicated in the prerequisites, you need to have a basic understanding of the structure of SOAP requests in XML format.
35
36 == Technical description ==
37
38 DigDash offers this functionality as a module (or add-on) to be installed with the product (it is not installed natively). There is therefore no "web service" type data source, but we will use the existing principle in DigDash of WLNK files to host the information needed to retrieve data via SOAP.
39
40 (% class="box infomessage" %)
41 (((
42 💡 A WLNK file in DigDash contains a URL. The content pointed to by the URL is downloaded by the server each time it needs it (for a data source or on request from a user).
43
44 //Why create a WLNK file?//
45
46 We could create a simple XML data source file, but this would simply be a snapshot of the data at a given point in time. The WLNK file makes it possible to refresh the data when necessary.
47
48 You should also bear in mind that a WLNK file only contains the information needed to call**a single SOAP request**.
49
50 ⇒ **One WLNK file = One SOAP request**
51 )))
52
53 In DigDash, you will not need to call the service entry point directly, but instead use a web application (which can be deployed on the same DigDash Tomcat server) that DigDash has developed and which is mainly the add-on. This web application will be responsible for sending the SOAP request with the information you have provided in DigDash and will be responsible for retrieving the SOAP response in DigDash.
54
55 [[image:SOAP_en.png||alt="SOAP" height="423" width="667"]]
56
57 = Installation and configuration of the SOAP module =
58
59 == Installation and deployment of the web application ==
60
61 1. First of all, stop the DigDash server.
62 1. Copy the **ddsoap.war** web application located in the **<digdash_installation>/add-ons/soap **folder to the **<digdash_installation>/apache-tomcat/webapps** folder.
63 1. Restart the DigDash server to deploy the application.
64
65 == wsdls folder ==
66
67 In the **<digdash_installation>/add-ons/ddsoap/wsdls** folder, place the list of WSDL files (.wsdl) corresponding to the descriptions of the services you wish to query.
68
69 == Configuring the soap_props.properties file ==
70
71 In the deployed application folder, configure the **ddsoap/soap_props.properties** file as follows:
72
73 === Miscellaneous properties ===
74
75 |=Properties|=Description
76 |=PRINT_DEBUG|(((
77 This property must be set to true for other traces to be displayed. Only error traces are displayed at all times.
78
79 This property is used to display debug traces in the logs.
80 )))
81 |=SHOW_REQUESTS|Used to display SOAP requests sent to the Web Service.
82 |=SHOW_RESPONSES|Displays SOAP responses sent by the Web Service.
83 |=LOGIN_BEFORE_CALL_SERVICE|Enables authentication to be performed before the request is sent, using the identifier properties (prefixed with **_creds **, see next section).
84 |=REMOVE_REQUEST_EMPTY_VALUES|Removes empty values (identified by "?") from the SOAP request.
85 |=REMOVE_REQUEST_EMPTY_NODES|Removes empty nodes from the SOAP request.
86
87 === Identifier properties ===
88
89 Authentication may be required before calling a service method.
90
91 **__ALL__ these properties must be prefixed with "creds_" __AND__ the LOGIN_BEFORE_CALL_SERVICE property must be set to "true" (see III.3.1).**
92
93 |=Properties|=Description
94 |(% colspan="2" %)**Warning **: special characters must be escaped in property names (\= for = , \: for {{{:)}}}
95 |**creds_binding**|(Mandatory) The name of the service used to authenticate.
96 |**creds_operation**|(Required) The name of the authentication operation.
97 |**creds_<username>**|The login, if any, for authentication.
98 |**creds_<password>**|The password, if any, for authentication.
99 |**creds_<any** other parameter>|Any other parameters used for authentication.
100
101 = Configuration in DigDash =
102
103 == Creating the WLNK file ==
104
105 We're going to start with the classic process of extracting data from a file in XML format.
106
107 1. From the **Templates** tab in the Studio, click on the **New Template** button.
108 1. In the **Create a new data model** box, click on the **All types **button in the **Files** section.
109 1. Select the document server you wish to use.
110 1. Click on the **Add a file...** button.
111 1. Three options are available. To retrieve content via SOAP, we'll use the **From a remote address or server** option
112 1*. The URL to call is that of the previously deployed **ddsoap** web application **(<digdash_server_address>/ddsoap**).
113 1*. The name of the WLNK file must end in **.xml**
114 1*. Check the option **Add only the URL link to the document server**. This option creates a WLNK file.
115 1*. You can customise the **maximum request time (in seconds)** for a WLNK (only values > 0 will be taken into account, otherwise the default timeout is applied).
116 [[image:ddsoap_file_en.png||alt="ddsoap wlnk file"]]
117
118 You can** enter **the parameters for the SOAP request in the form of a key-value in the GET HTTP parameters:
119
120 * Click on the **HTTP GET parameters... **button, then enter the name and value of each parameter.
121
122 **__You__ MUST enter at least these two parameters **:
123 _binding: the name of the Service to be called
124 _operation: the name of the operation or method of the Service to be called.
125
126
127 [[image:ddsoap_HTTP_request_parameters_en.png||alt="HTTP request parameters"]]
128
129 For the other SOAP parameters, you must specify their names (don't forget the namespaces if there are any). If several parameters appear in the request, fill in the names of the parents enabling you to reach the required node.
130
131 In general, the first node in the request with the specified name will be selected, unless the path to a specific node is specified (by going back into the parents, but not all the way to the root).
132
133 |**Separators**|**Description**|**Example**
134 |/|Path separator in the XML tree.|(((
135 <a>
136
137 <a1>
138
139 <b>?</b>
140
141 **<c>?</c>**
142
143 </a1>
144
145 <b>?</b>
146
147 **<c>?</c>**
148
149 </a>
150
151 <a1>
152
153 <b>?</b>
154
155 <c>?</c>
156
157 </a1>
158
159 **To access element c, write a/c (c will select the first element c)**
160 )))
161 |@|To access an attribute of an XML element.|(((
162 <a>
163
164 <b>?</b>
165
166 <c **attr=?>**?</c>
167
168 <a>
169
170 **To access the attr attribute of element c, we would write a/c@attr (or c@attr[[path:mailto:c@attr]] directly)**
171 )))
172
173 Once the SOAP parameters have been entered, click **OK** to validate.
174 ➡ The WLNK document will be created and uploaded to the selected document server.
175
176 This WLNK file can then be loaded as an XML data source in the conventional way.
177
178 (% class="box infomessage" %)
179 (((
180 The newly created WLNK file can then be modified manually if required. Right-click on the "Configure WLNK file" menu to edit the link and query parameters.
181 [[image:1768225549437-542.png||alt="Contextual menu"]]
182
183 [[image:configure_wlnk_file_en.png||alt="Edit wlnk"]]
184 )))
185
186 == Pagination management ==
187
188 The SOAP module can help you aggregate the data from a paginated request to retrieve all the data from DigDash.
189
190 To do this, you **must** specify the following GET parameters in the URL:
191
192 **pagination**: this parameter must be true to specify a case of pagination.
193
194 **_pageTag**: in this parameter you must specify the name of the tag (it can be an element or an attribute) which defines pagination (see the table: //Naming convention for accessing a node //to define this parameter).
195
196 **_pageStartElement**: this is the name of the element containing the lines of data we are interested in (see next section). If you do not know the structure of the SOAP response, you will generally have to send an initial request without pagination to find out the name of this tag (see the table: //Naming convention for accessing a node //to define this parameter).
197
198 **pageInc**: the page index starts at 0. This parameter defines the incrementation of this index.
199
200 //_resultSize// (optional): specifying the total number of results optimises response time.
201
202 == Configuring the XML data source ==
203
204 Loading the SOAP response in XML format as a data source in DigDash can be difficult to read at first.
205
206 You need to make a few changes to the configuration of the DigDash data source to make it easier to read.
207
208 For example: if we consider a SOAP response with the following XML structure :
209
210 {{code language="XML" cssClass="notranslate"}}
211 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
212 <soapenv:Body>
213
214 <a></a>
215
216 <structs>
217 <struct1>
218 <s11></s11>
219 <s12>
220 <s121></s121>
221 </s12>
222 <s13></s13>
223 <s14></s14>
224 </struct1>
225
226 <struct2>
227 <s21></s21>
228 <s22>
229 <s221></s221>
230 </s22>
231 <s23></s23>
232 <s24></s24>
233 </struct2>
234
235 <struct3>
236 <s31></s31>
237 <s32>
238 <s321></s321>
239 </s32>
240 <s33></s33>
241 <s34></s34>
242 </struct3>
243 </structs>
244
245 </soapenv:Body>
246 </soapenv:Envelope>
247 {{/code}}
248
249 (((
250 We need to try and identify the structure encompassing the structures (rows) we are interested in. In this example of a SOAP response, the structures //struct1//, //struct2//, //struct3// are the ones we are interested in.
251 )))
252
253 The structure encompassing these is the //structs// structure //.//
254
255 In the configuration of the data model, in the advanced options for data selection, you will therefore need to specify the following:
256
257 1. Click on the **Advanced** button in the **Data selection** section.
258 [[image:1768226357185-193.png||alt="Avancé"]]​​​​​​​
259 1. (((
260 Check the **Start reading XML from a specified tag** option and specify the structure from which to start reading the XML.
261 It is also preferable to uncheck the **Expand child nodes and attributes into columns** option
262 )))
263 1. (((
264 Click **OK**.
265 )))
266
267 = Glossary{{id name="Lexique"/}} =
268
269 In this document we refer to :
270
271 * **SOAP**: Simple Object Access Protocol. It is a protocol for exchanging structured information in the implementation of XML-based web services. (Wikipedia)
272 * **WSDL**: Web Services Description Language. This is a file in XML format used to describe a web service.
273 * **WLNK **: Web LiNK. Files with the .wlnk extension contain links to data sources.
274
275 = References =
276
277 //DigDash uses the soapui OpenSource library from SmartBear Software to support exchanges with SOAP web services.//
278
279 [[https:~~/~~/www.soapui.org>>url:https://www.soapui.org/]]