Recipe 9.5 Performing a Bulk Insert with SQL Server

Problem

Given many records in an XML file that you need to add to a SQL Server 2000 database, you need to perform a bulk insert with optimal performance.

Solution

Perform a fast bulk insert and update using the XML bulk load functionality in Microsoft SQL Server 2000.

You'll need a reference to the Microsoft SQLXML BulkLoad 3.0 Type Library from the COM tab in Visual Studio .NET's Add Reference Dialog.

The sample uses a single XSD file:

Customers.xsd

The schema for the data that is bulk loaded into the Customers table

The sample uses a single XML file:

Customers.xml

Contains the data that is bulk loaded into the Customers table

The sample code creates a bulk load object SQLXMLBulkLoad and sets the connection string and error log file for the object. The Execute( ) method of the SQLXMLBulkLoad object is used to bulk load the Customers data in the XML file into the Customers table in the Northwind database. The Customers table must be empty prior to running this sample, otherwise, a primary key constraint error will be raised and written to the error log.

The Customers XSD file is shown in Example 9-7, and the XML file is shown in Example 9-8.

Example 9-7. File: Customers.xsd

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
    <xsd:element name="ROOT" sql:is-constant="true">
        <xsd:complexType>
            <xsd:sequence>
                <xsd:element ref="Customers" />
            </xsd:sequence>
        </xsd:complexType>
    </xsd:element>
    <xsd:element name="Customers" sql:relation="Customers">
        <xsd:complexType>
            <xsd:sequence>
                <xsd:element name="CustomerID" type="xsd:string"
                    sql:datatype="nvarchar(5)" />
                <xsd:element name="CompanyName" type="xsd:string"
                    sql:datatype="nvarchar(40)" />
                <xsd:element name="ContactName" type="xsd:string"
                    sql:datatype="nvarchar(30)" />
                <xsd:element name="ContactTitle"
                    type="xsd:string"
                    sql:datatype="nvarchar(30)" />
                <xsd:element name="Address" type="xsd:string"
                    sql:datatype="nvarchar(60)" />
                <xsd:element name="City" type="xsd:string"
                    sql:datatype="nvarchar(15)" />
                <xsd:element name="Region" type="xsd:string"
                    sql:datatype="nvarchar(15)" />
                <xsd:element name="PostalCode" type="xsd:string"
                    sql:datatype="nvarchar(10)" />
                <xsd:element name="Country" type="xsd:string"
                    sql:datatype="nvarchar(15)" />
                <xsd:element name="Phone" type="xsd:string"
                    sql:datatype="nvarchar(24)" />
                <xsd:element name="Fax" type="xsd:string"
                    sql:datatype="nvarchar(24)" />
            </xsd:sequence>
        </xsd:complexType>
    </xsd:element>
</xsd:schema>

Example 9-8. File: Customers.xml

<ROOT>
    <Customers>
        <CustomerID>ALFKI</CustomerID>
        <CompanyName>Alfreds Futterkiste</CompanyName>
        <ContactName>Maria Anders</ContactName>
        <ContactTitle>Sales Representative</ContactTitle>
        <Address>Obere Str. 57</Address>
        <City>Berlin</City>
        <PostalCode>12209</PostalCode>
        <Country>Germany</Country>
        <Phone>030-0074321</Phone>
        <Fax>030-0076545</Fax>
    </Customers>

<!--  . . .  -->

    <Customers>
        <CustomerID>WOLZA</CustomerID>
        <CompanyName>Wolski  Zajazd</CompanyName>
        <ContactName>Zbyszek Piestrzeniewicz</ContactName>
        <ContactTitle>Owner</ContactTitle>
        <Address>ul. Filtrowa 68</Address>
        <City>Warszawa</City>
        <PostalCode>01-012</PostalCode>
        <Country>Poland</Country>
        <Phone>(26) 642-7012</Phone>
        <Fax>(26) 642-7012</Fax>
    </Customers>
</ROOT>

The C# code is shown in Example 9-9.

Example 9-9. File: BulkInsertForm.cs

// Namespaces, variables, and constants
using System;
using System.Configuration;
using System.Windows.Forms;
using SQLXMLBULKLOADLib;
using System.Data;
using System.Data.SqlClient;

private const String DATAFILENAME =
    ConfigurationSettings.AppSettings["Project_Directory"] +
    @"Chapter 09\Customers.xml";
private const String SCHEMAFILENAME =
    ConfigurationSettings.AppSettings["Project_Directory"] +
    @"Chapter 09\Customers.xsd";
private const String ERRORLOGFILENAME =
    ConfigurationSettings.AppSettings["Temp_Directory"] +
    "BulkLoadError.log";

//  . . . 

// Create the bulk load object, defining connection, and error log.
SQLXMLBulkLoad bl = new SQLXMLBulkLoad( );
bl.ConnectionString =
    ConfigurationSettings.AppSettings["OleDb_Msde_ConnectString"];
bl.ErrorLogFile = ERRORLOGFILENAME;

// Execute the bulk load.
try
{
    bl.Execute(SCHEMAFILENAME, DATAFILENAME);
    MessageBox.Show("Bulk load completed successfully.", "Bulk Load",
        MessageBoxButtons.OK, MessageBoxIcon.Information);
}
catch (Exception)
{
    MessageBox.Show("ERROR. See " + ERRORLOGFILENAME + " for details.",
        "Bulk Load Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
finally
{
    bl = null;
}

Discussion

The SQL Server XML Bulk Load component is used through COM interop to bulk insert data contained in a XML document into a SQL Server database. This component controls the execution of a XML bulk load operation. The example defines an optional error log file, where the default is an empty string meaning that no error log is created.

You can bulk load data into multiple parent-child tables at the same time, a feature that is not available in the OpenXML Transact-SQL extension.

For information about the XML Bulk Load component and its methods and properties, see the topic "XML Bulk Load" in the MSDN Library.