When dealing with a large amount of data or migrating workflow configuration, we might need to generate and use INSERT script. This article describes how to do it with SQL Server database.
Why do we need to generate INSERT script?
When deploying changes, the preferred method is DBC script. There are different tools to generate DBC scripts for many types of configurations. However, in certain cases, such as when migrating workflow configuration, we still need to use SQL script.
For deploying a data package, it is a good practice to script and deploy the change as part of the DBC deployment process. It is easier to track the changes with version control system such as GIT or SVN).
As an example, in a recent project, the users used Maximo in DEV to enter and validate the data. After they finished, I extracted the data using the method described in this post and loaded it to Production as part of the DBC deployment process.
Do you know the UpdateDB or RunScriptFile tool can deploy SQL files directly? It is not required to put SQL statements into a DBC file with the FREEFORM tag.What is the limitation with SQL Server Management Studio?
With Oracle database, generating INSERT statements is quite simple with SQL Developer. All we need to do is right clicking on a table, then choose Export Data > Insert. However, with SQL Server, a similar function is buried deep in the drop-down menu and is hard to find. There are at least two other features with similar purpose in the same menu which causes confusion and makes people thought this is not possible with SQL Server. These functions don’t work for this requirement. They are:
- Right-click on the table, then choose Script Table As > INSERT to
- Right-click on the database, then choose Tasks > Export Data
For this requirement, we need to right-click on the database (not table), then choose Tasks > Generate Scripts…
However, this approach has some limitations:
- It does not let you define a WHERE clause. You’ll need to generate script for the whole table.
- With Maximo, all tables have the ROWSTAMP field which you need to exclude. Otherwise, the INSERT script does not work.
How to generate INSERT Script?
To address the limitations described above, we can create a VIEW or a staging TABLE from the source table. This allows us to set a WHERE clause to filter the data we want to extract and exclude the ROWSTAMP column. A VIEW is more convenient in a development project because we can define any rule to manipulate the data such as changing the ID field to a new range to avoid conflict. However, note that there is a bug with older versions of SQL Servers that it cannot generate script correctly for the Data Only mode. In such case, we can use a staging table instead.
To provide an example, below are the detailed steps to generate INSERT script for all “Fire Extinguisher” records in the ASSET table:
Step 1: Create a temporary table:
- Use one of the two SQL statements below to get a comma-separated list of column names from the table. The first query uses the MAXATTRIBUTE table, and it only works for Maximo database. The second query works for any generic SQL database, but you will need the SELECT permission to the Information_Schema
SELECT string_agg(column_name, ', ')
WITHIN GROUP (ORDER BY column_name)
FROM information_schema.columns
WHERE table_name = 'asset' AND column_name != 'rowstamp'SELECT string_agg(attributename, ', ')
WITHIN GROUP (ORDER BY attributeno)
FROM maxattribute WHERE objectname = 'asset'
AND persistent = 1- Copy/Paste the list of columns to write a CREATE TABLE statement below, then execute it to create the table. In this step, we can also transform or map any ID fields (such as ASSETID and ASSETUID in this example) to avoid conflict.
DROP TABLE IF EXISTS tmp_asset;
SELECT [Copy/Paste the column names here]
INTO tmp_asset
FROM asset
WHERE description = 'Fire Extinguisher';Step 2: Generate INSERT Script from the temp table:
- Right-click on the database, choose Tasks > Generate Scripts…
- In the Generate Scripts wizard, choose “Select specific database objects”, then select the tmp_asset table we created in the step above.
- In the next page, “Set Scripting Options”, click on the “Advanced” button, under the General group, change the “Types of data to script” option from Schema Only to Data Only
- Back to the Set Scripting Options, either choose to save the output as a SQL file or open it in a new query window.
What are the alternative methods?
If the method described in this post does not work for you, two alternative methods you can try are
- Use the GENINSERTDBC tool
- Use Microsoft Visual Studio to generate INSERT scripts
