Archive for category SQL Server

SSIS Consume Web Service with Complex Types (GAC Avoided solution)

This blog explains the easiest method SSIS can consume web service methods consist of complex types as the request or response parameters.

If a web service method consist of complex types as the request or response parameters, we can’t use the build-in Web Service Task. Although, there are posts online where we can consume it programmatically within VSA (Script Task).

Since we can’t directly add Web Reference from the VSA editor, some people suggested to build a separate library project that make the web service calls, and let the VGA from SSIS reference this library. The problem with this solution is that the built library would have to be registered to GAC on server running the SSIS package. This may sometimes be policatically difficult epecially when the package is being executed by SQL Server Agent. We wouldn’t want to install library to GAC on a SQL Server machine for just making a web service call.

In order to avoid going down the path to touch GAC, what we can do is to use the WSDL.exe tool to generate the proxy class and import this class into the Script Task code, and consume the web service from there.

Proxy Class Generation

  1. Please open Visual Studio Command Prompt. It should located under “Visual Studio Tools” subfolder in the Start Menu.
  2. cd into an empty directory and type in the following command
    • wsdl http://Domain/WebServiceURL?wsdl /l:vb

Import Proxy Class to VSA editor

  1. Add a new Script Task, and go into the Design Script window
  2. Add a new class under the Script Task project
  3. Copy and paste all the content from the generated class to this new added class file
  4. This is optional, you may wrap this class into a namespace to make it cleaner.

Consuming the service

We may now consume the web service by instantiating the client object. In order for the script knows which endpoint the client is calling, you may need to set the end point manually like this:

, , , ,

No Comments

SQL Server Integration Service Package Elevation Tips

When developing SSIS package, very often, the development environment is different from the testing environment as well as the production environment. Different environment would probably have different connection strings for different connections. They can be database connection string, flat file location path etc. To make elevation easier, connection information under Connection Manager are designed to be configurable when the package is being registered under SQL Agent. Moreover, initial value of the variables can also can configure when attaching the package as part of the SQL job.

In the blog, I’ll show you some How to deploy SSIS packages for SQL Agent to execute with custom configuration.
I’ve created a simple SSIS package that take a global variable “env” and write it to a csv file along with a timestamp obtain from a database. Here’s a copy of the dtsx for download: demo.dtsx

Job Creation

When creating a job to execute a SSIS package, you can either use the GUI from SQL Server Management Studio or by SQL query. I usually use the graphic interface first, then extract it as a SQL script and modify it from there. Let’s try it out together!

Open Microsoft SQL Server Management Studio and connect to the database instance where you want the job to be located. Once connected, under SQL Server Agent, right click on jobs and choose to create a new job. After entering a job name and the owner name, hit “Step” on the left menu bar and click on the “New” button for a new step.

On the Job Step Property screen, choose SQL Server Integration Service Package for type and File System for package source. Note that we’re using File System in this demo for simplicity, however, you may save the package into the ssis package store on SQL Server and select the package from there.

ssisDemo_001

My database instance is localhost, so I simply reference the package as local file on the system. If the database instance you connects to is not your localhost, you can simply create a network share and reference your dtsx file like: \\yourhostname\shareName\demo.dtsx.

Please note that the job we’re setting right now is purely for development purpose, and the location of the package can be modify when we deploy to another environment.

After setting the General tab, let’s move onto the Data sources tab.

ssisDemo_002

In this tab, you can specify connection strings for each of the connection managers in the package. These are the connection information SQL agent is going to use when executing the package. Since we’re creating this job for development testing, let’s select the two connection managers and we’ll customize the values when we generate the elevation scripts for testing and production environments.

In this SSIS package, a global variable named “env” is set to “DEV” by default, we’ll specify the value once again on the “Set Values” tab for the elevation script.

ssisDemo_005

Please take a note on the syntax: \package.variables[env].Value = “DEV”

Now we can press OK twice to save the job.

We may try running the SQL job manually to see if it runs ok. The next step for us is to prepare elevation script to different environments. Let’s right click on the job, and generate the SQL script used to create this job.

The script would look something like this:

USE [msdb]
GO
/****** Object:  Job [testJob]    Script Date: 09/15/2009 01:19:13 ******/
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
/****** Object:  JobCategory [[Uncategorized (Local)]]]    Script Date: 09/15/2009 01:19:13 ******/
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

END

DECLARE @jobId BINARY(16)
EXEC @ReturnCode =  msdb.dbo.sp_add_job @job_name=N'testJob',
		@enabled=1,
		@notify_level_eventlog=0,
		@notify_level_email=0,
		@notify_level_netsend=0,
		@notify_level_page=0,
		@delete_level=0,
		@description=N'No description available.',
		@category_name=N'[Uncategorized (Local)]',
		@owner_login_name=N'DC\Jack Wong', @job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
/****** Object:  Step [Execute demo SSIS]    Script Date: 09/15/2009 01:19:14 ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Execute demo SSIS',
		@step_id=1,
		@cmdexec_success_code=0,
		@on_success_action=1,
		@on_success_step_id=0,
		@on_fail_action=2,
		@on_fail_step_id=0,
		@retry_attempts=0,
		@retry_interval=0,
		@os_run_priority=0, @subsystem=N'SSIS',
		@command=N'/FILE "C:\projects\ssisVariableDemo\ssisVariableDemo\bin\demo.dtsx" /CONNECTION "localhost.master";"Data Source=localhost;Initial Catalog=master;Provider=SQLNCLI.1;Integrated Security=SSPI;Auto Translate=False;" /CONNECTION result;"c:\result.csv" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /SET "\package.variables[env].Value";"""DEV""" /REPORTING E',
		@database_name=N'master',
		@flags=0
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
    IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:

For elevation to different environment, we’d probably need to modify the connection manager strings and the default value for the global variable “env”. Also, the owner of the job would probably need to be modified as well.

On line 25, you may update the @owner_login_name to set the owner of the job

On line 38, the @command parameter is to specify any runtime specific customize values and configurations. There are three things you can modify.

  • The ssis package source: you may specify the source type and location. Thus your ssis package can be elevated and staged and needs no changes for different environments
  • Connection Manager – connection strings: You may specify a new location of the csv file and/or a new database with user credential
  • Variable values: You may specify initial values for any variables within the package.

This is it for this blog. I hope it can be helpful to you.

, , , , , , ,

No Comments