geohernandez
Menu
  • HOME
  • ABOUT
  • CONTACT ME
  • WORK WITH GEO
    • Data Specialist
    • Speaker Events
    • Resume
  • English
    • English
    • Español
Menu

How to import a Flat File into a SQL Server database using SSIS

Posted on August 7, 2018September 16, 2018 by geohernandez

The purpose of this article is to show you how you can use SSIS for doing a classic task – for example, importing a Flat File into a SQL Server work table.  This article is a step by step guide and the first one in a series.

For this article we are going to use the fictitious DB called TSQLRaceDB, the first part consists of a simple process to import directly into the User table.


CREATE DATABASE TSQLRACE;
GO

CREATE SCHEMA Admin;
GO

CREATE TABLE Admin.Account
(
AccountID INT IDENTITY
NOT NULL
PRIMARY KEY ,
Description NVARCHAR(40) NULL ,
RegUserId INT NOT NULL ,
RegDate DATETIME2(2) NOT NULL
);

We start with the creation of a new project in SSIS, we open Visual Studio and choose the SSIS Project.

The first thing I always do in an SSIS project is to define the Connections, you should right click on Connection Managers and then on New Connection Manager.

We will immediately see a group of Connections sources, in this case, we choose the Flat file type.

I then fill the Connection manager name and a brief description about it, later you should click the Browse button, in this first part I am going to localize the CSV, however, in the next part of this article I would like to teach you how to dynamically set up this property, meanwhile, we are going to use a hardcoded file path.

We can see that we are using a simple CSV file called Accounts.csv, it has a small number of rows, I realized from my experience that in general, for a small file, for instance, less than 100 MB, this proposed design is very effective and faster than other options.

The next step is to configure the columns and the data type captured from a file, in this stage you should adjust the data type according to the target table, it’s not mandatory because you can add a specific step to do it through a DataConversion task.  There is a useful option in which SSIS will automatically define the data type based on the values that are in the column. However, if incorrect, it can be manually modified in this section.

When you click in the Preview tab then you can observe a preview of the file contents.

One time that we have created the Flat File connection the next step is to choose the Data Flow task from the Toolbox.

On the next window, we double click and we are in Data Flow section, here we can take a source assistant connection and choose the Flat File type because this is the only connection configured in the project.

At this point, we have a valid source and the first step has finished, but we need to move the information toward a final table, to do this we will need to configure the target connection, then we click the Connection Managers section and we should choose an OLEDB Connection to the SQL Server instance and point out to the target DB.

After testing the connection, we can return to the Data Flow and drag and drop the Destination Assistant. In this case, we should choose SQL Server and the connection which has recently been built.  We should right click and set up the target table available for the OLEDB connection.

In the tab Mapping, we are going to map the columns from Source to Target, it’s a simple process where we associate each column.

For testing the result, we only need to click the Start button and we can observe the workflow and each of the components with a green ticker and the number of rows processed.

After this, we have reached our goal but…. there is something missing because the path of the CSV file is pointing out to my computer and this is not a realistic scenario, what do we do? We should create this path file in a configurable way independent of the Environment – for example, Development, UAT and Production.  I am going to apply the approach which consists in the use of a Project Parameter.  We have to double click on Projec.params.

We then click on the icon  to add a new parameter and write the path of the CSV file.

The next step is to parameterize the path into CSV Flat File Connection, we choose the connection and right-click and next, the Parameterize option.

We should associate the property ConnectionString with the newly defined project parameter.

What is the next step? One time that we have defined the Path of CSV file we will be able to associate it to the Environment Variables in the SSIS Server instance, how to deploy an SSIS package is out the scope of this article, however, at the end of this article, I am posting some links related with this topic.

Following our example:

I right-click and define a new variable

Later we have to link the Project Parameter and path file CSV variable, the next step is to right-click on Projects and Configure.

 

Immediately we can see the next window where we have to link Project.parameter and Variable, but first we have to click on the References page and  the Add button.

Later, we click on the Parameters page and choose the Value for the variable which was previously defined.

Once that we have configured this option we will be able to run the package and it is going to take the path defined in the environment variable.

Conclusion

I hope you will find this article useful and it will become a good starting point for developing more complex solutions which involve the process of working with flat files in SSIS.

 References

For more information, refer to these links:

https://docs.microsoft.com/en-us/sql/integration-services/packages/run-integration-services-ssis-packages

https://docs.microsoft.com/en-us/sql/integration-services/packages/run-a-package-in-sql-server-data-tools

Category: SSIS

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search for articles

Recent Posts

  • Quick Guide: BigQuery Service Account Setup Using gcloud
  • The Art of Data Modeling in AI times
  • Getting Started with Snowflake’s Snowpipe for Data Ingestion on Azure

Categories

  • Airflow (1)
  • Azure (6)
  • Azure DevOps (2)
  • Bash script (1)
  • Blog (1)
  • Cassandra (3)
  • Chronicles from the trenches (26)
  • Data Architecture (3)
  • Data Engineering (11)
  • DB optimization (2)
  • Events (2)
  • GIT (1)
  • MySQL (1)
  • Python (7)
  • Snowflake (3)
  • SQL Saturday (1)
  • SSIS (2)
  • T-SQL (5)
  • Uncategorized (2)

Archives

  • May 2025 (1)
  • March 2025 (1)
  • January 2025 (2)
  • October 2024 (1)
  • July 2024 (1)
  • May 2024 (1)
  • December 2023 (1)
  • November 2023 (1)
  • August 2023 (1)
  • June 2023 (1)
  • December 2022 (1)
  • November 2022 (1)
  • July 2022 (1)
  • March 2022 (1)
  • September 2021 (1)
  • May 2021 (1)
  • March 2021 (1)
  • February 2021 (3)
  • December 2020 (1)
  • October 2020 (3)
  • September 2020 (1)
  • August 2020 (1)
  • January 2020 (1)
  • August 2019 (1)
  • July 2019 (1)
  • June 2019 (1)
  • May 2019 (1)
  • April 2019 (1)
  • March 2019 (1)
  • November 2018 (3)
  • October 2018 (1)
  • September 2018 (1)
  • August 2018 (2)
© 2025 geohernandez | Powered by Minimalist Blog WordPress Theme