Follow

Remote Onsite Integration with a SaaS Instance

Overview

The SD Elements integration process normally requires direct network access with an external server. When SD Elements is hosted as SaaS this requirement is not an issue if the external server is also hosted on the Internet. If, however, the external ALM or Scanner server is hosted inside a private network - this requirement cannot be met. To address this situation - where the external server is inaccessible to SD Elements - a workaround is provided: the Remote Integration Client.

The following image illustrates the typical configuration using the Remote Integration Client:

  • SD Elements instance hosted in the Internet
  • The Remote Integration Client running inside a private network. The client has network access to both SD Elements and the internal ALM server (Jira, Rally, Team Foundation Server, etc.)

To allow integration between SD Elements (SaaS) and a private ALM or Scanner Tool server,

  1. The server connector should be marked "inaccessible" in "System > Integration"
  2. Install the Remote Integration Client on a computer having network access to the SD Elements SaaS server and the private server.

Once the server connector has been added to SD Elements, project teams can setup a project integration with the ALM or Scanner Tool using the web UI as normal, including setting its frequency.

Part of any project integration setup is configuring how often the integration should run: manually, hourly, daily, weekly, monthly. The Remote Integration Client can be configured to run on a similar frequency.

NOTE: At the moment one caveat of remote integration is "manual" frequency is unavailable to remote connections.

When the Remote Integration Client runs it communicates securely over https to the SD Elements API while it executes the following steps:

  • Authenticates to the server as a user using an API token
  • Retrieves the list of project integration jobs that the user has permission to view and affect.
  • For each integration job its full details are retrieved from the SD Elements server; including server address and credentials
  • Each job is executed by the client and the job's result (success or failure) is posted back to the SD Elements server.

User Permission Requirements

An SD Elements user running the integration onsite must have:

- Global permissions "Edit Alm connections" to run jobs that sync with Alm, and "Edit security tool connections" to run jobs that import scanner results.

- For each project, the user should have "Sync with Alm tools", and "Verify tasks" permissions.

Remote Integration Client Requirements

- Linux, Mac OS X or Windows

- HTTPS network access to the SD Elements SaaS server

- Network access to the private server

Installation

Windows

Installing the Remote Integration Client on Windows is a straight-forward process.

Linux or Mac OS X

Two requirements must be met to download and run the client:  

  • Python 2.6 or 2.7
  • git installed and network access to https://github.com/

In a terminal session run:

git clone https://github.com/sdelements/sdetools.git
cd sdetools
git checkout master
pip install -r requirements.txt

Running Instructions

Make sure you have the permissions listed above and an APIv2_CONNECTION_STRING.

To run all integration work accessible by an SD Elements user, run:

python sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING

Use different command-line options to filter integration jobs; including frequency and projects. Refer to the help for more options:

python sde.py help command_driver

Examples:

1. Use the command-line option --filter_frequency to cron jobs that match the job frequencies in the web interface: manually, hourly, daily, weekly, monthly

python sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --filter_frequency=monthly

2. Target a specific project "Demo Project" in SD Elements and run its integration jobs using --filter_params (accepts a JSON dictionary of key-value pairs). The list of available parameters are dependent on jobs. "sde_businessunit", "sde_application" and "sde_project" are always available.

python sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --filter_params="{\"sde_project\":\"Demo Project\"}"

Troubleshooting

Assuming a user has the proper project permissions any errors that occur during an integration will display on the web interface. Further information about an integration run can be found by adding "-v" (verbose) or "-d" (debug) command-line arguments.

Verbose logging:

python sde.py command_driver -v --sde_api_token=APIv2_CONNECTION_STRING

Debug-level logging (prints out the full set of parameters for an integration including credentials):

python sde.py command_driver -d --sde_api_token=APIv2_CONNECTION_STRING

Detailed API Logging:

python sde.py command_driver -d --debugmods=sdetools.sdelib.restclient --sde_api_token=APIv2_CONNECTION_STRING

Trust a new SSL Certificate:

python sde.py add_ssl_cert --server=SERVER.COM --port=443

Disable SSL certification validation for ALM Integration:

python sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --command_params="{\"alm_validate_cert\":\"False\"}"

Run a specific integration for an ALM or Scanner:

python sde.py command_driver --filter_connections=alm-XX --sde_api_token=APIv2_CONNECTION_STRING

Where XX is a database ID. This value is not shown on the Web UI, but you can get it from the project connection links.

On a Project > Integration page there will be a list of connections with a "Sync" button. If you were to click the button you'd start syncing the tasks over to an ALM (Jira or Rally or ...) Examine the links of each connection. These links have a form similar to:

https://your.server.com/bunits/general/demo-application/demo-project/integration/alm/335

When you find the integration you want to run, examine its URL. It should look similar to the one above. In this example you would use 335 for XX.

Connecting through a Proxy

The Remote Integration Console can normally detect when it should connect through a proxy server. In cases where it cannot, you can follow the steps below:

Windows

Open "integrate.bat" which is located in "C:\Users\YOUR-NAME\AppData\Local\Programs\SD Elements Remote Integration\integrate.bat" and add the proxy's server and port to the top of the file:

SET http_proxy=http://someproxy.com:3128

Linux or Mac OS X

Before calling the "python sde.py" process, set the "http_proxy" environment variable to the proxy's server and port:

export http_proxy=http://someproxy.com:3128

Scheduling Ongoing Integration

By using the filtering options outlined above it is possible to schedule on-going integration to match the SD Elements user-interface.

On Unix-like environments the following crontab can be used to run jobs consistent with the UI. In the terminal type

crontab -e

and paste the following crontab,

# Hourly (On the hour)
0 * * * * python /path/to/sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --filter_frequency=hourly > /path/to/output.log 2> /path/to/debug.log
# Daily (Midnight)
0 0 * * * python /path/to/sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --filter_frequency=daily > /path/to/output.log 2> /path/to/debug.log
# Weekly (Mondays)
* * * * 1 python /path/to/sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --filter_frequency=weekly > /path/to/output.log 2> /path/to/debug.log
# Monthly (First of the month)
0 0 1 * * python /path/to/sde.py command_driver --sde_api_token=APIv2_CONNECTION_STRING --filter_frequency=monthly > /path/to/output.log 2> /path/to/debug.log

Update the paths to reflect the relevant locations.

On Windows, the installer will prompt a user on whether to automatically create Windows Task Scheduler entries for hourly, daily, weekly and monthly frequencies.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments