Deborah's Developer MindScape






         Tips and Techniques for Web and .NET developers.

August 25, 2009

Text Files

Filed under: C#,Text Files,VB.NET,XML @ 1:09 pm

There are many different types of text files that you may need to process in your applications. Some of the more common types are described in this post.

Delimited files

Delimited files separate the fields of the file with some type of a delimiter. The most common delimited files are comma separated value (CSV) files, as shown below.

1,  Baggins, Bilbo, 20090811
2,  Baggins, Frodo, 20090801
3,  Gamgee,  Samwise, 20090820
4,  Cotton,  Rosie, 20090821

Other field delimiters, such as tabs or semicolons, can also be used.

In most cases, delimited files define the end of each record with a CrLf (carriage return/linefeed). So the example above contains four records, one for each line of data.

If you need to read the data from a delimited file into your application, there are features in .NET to assist you.

You can read the file directly using the file classes within .NET and then process the columns using String functions.

Alternatively, you can read the file into a DataTable, even if you have no plans to use this data in a database. This allows you to work with the data as a set of rows and columns instead of using a large number of string manipulation functions. See this link for an example of reading a CSV text file into an in-memory DataTable.

Another option is to read the file using VB’s TextFieldParser class. See this link for an example.

Fixed length files

Fixed length files define data within specific columns. In the example below, the customer Id is left-justified in the first 8 columns, the last name is left-justified in the next 20 columns, the first name is left-justified in the next 10 columns, and the last edit date is left justified in the last 10 columns.

000001  Baggins             Bilbo     20090811
000002  Baggins             Frodo     20090801
000003  Gamgee              Samwise   20090820
000004  Cotton              Rosie     20090821

As with delimited files, the end of each record is normally defined with a CrLf (carriage return/linefeed). So the example above contains four records, one for each line of data.

This style of text file is often used by legacy systems, especially mainframe systems.

The StringBuilder class in the .NET framework provides easy to use formatting features that help you write code to output fixed length files. See this link for an example.

If you need to read the data from a delimited file into your application, there are features in .NET to assist you.

You can read the file directly using the file classes within .NET and then process the columns using String functions.

Alternatively, you can read the file into a DataTable, even if you have no plans to use this data in a database. This allows you to work with the data as a set of rows and columns instead of using a large number of string manipulation functions. See this link for an example of reading a fixed length file into an in-memory DataTable.

Another option is to read the file using VB’s TextFieldParser class. See this link for an example.

XML files

XML (eXtensible Markup Language) files follow the XML specification to define hierarchical data. You use XML elements, defined with start (< >) and end (</ >) tags and XML attributes, define with name/value pairs within a start element tag, to define the structure and content of a file.

For example, this XML file defines a set of customers, each delimited within customer tags. Each field defined for the customer is defined in an element within the customer open and close tags. In the example below, each customer has a CustomerId, LastName, FirstName, and LastUpdateData.

<customers>
  <customer>
    <CustomerId>1</CustomerId>
    <LastName>Baggins</LastName>
    <FirstName>Billbo</FirstName>
    <LastUpdateDate>20090811</LastUpdateDate>
  </customer>
  <customer>
    <CustomerId>2</CustomerId>
    <LastName>Baggins</LastName>
    <FirstName>Frodo</FirstName>
    <LastUpdateDate>20090801</LastUpdateDate>
  </customer>
  <customer>
    <CustomerId>3</CustomerId>
    <LastName>Gamgee</LastName>
    <FirstName>Samwise</FirstName>
    <LastUpdateDate>20090820</LastUpdateDate>
  </customer>
  <customer>
    <CustomerId>4</CustomerId>
    <LastName>Cotton</LastName>
    <FirstName>Rosie</FirstName>
    <LastUpdateDate>20090821</LastUpdateDate>
  </customer>
</customers>

If you have any choice on the type of text file to use for your application, an XML file is the best choice. XML has become the standard for defining data structures due to its simplicity, clarity, self-descriptiveness, and consistency. There are great tools for reading and writing XML files available in the .NET framework. Visual Basic has additional features, called XML literals, that make working with XML files a breeze. See this link for other posting on working with XML files.

Sometimes, however, you have no choice. You have to read a text file that is not in an XML structure. For example, you may need to read files generated by legacy systems or by other software. In that case, you need to use delimited or fixed length files.

6 Comments

  1.   Neil — August 25, 2009 @ 2:04 pm    Reply

    No mention of JSON?

  2.   DeborahK — August 25, 2009 @ 3:45 pm    Reply

    Hi Neil –

    Thank you for stopping by my blog.

    I was focused on describing enough about text files to then write the next two posts, which involve reading text files using OleDb.

    You would not use OleDb to read in JSON files. But for completeness, I’m glad you mentioned it here.

    Thanks again!

  3.   Waleed El-Badry — August 25, 2009 @ 7:16 pm    Reply

    Thanks Deborah for this post.

    Is there a chance to see a post related to interacting with Excel from VB? I’m not talking about VSTO but using automation.

    Regards

  4.   DeborahK — August 25, 2009 @ 7:34 pm    Reply

    Hi Walleed –

    I have done several Excel examples:

    http://msmvps.com/blogs/deborahk/archive/tags/Excel/default.aspx

    Hope this helps.

  5.   Elyza — August 13, 2011 @ 12:28 pm    Reply

    And I was just wodnernig about that too!

  6.   Kris — August 14, 2011 @ 10:24 am    Reply

    That saves me. Thanks for being so senbslie!

RSS feed for comments on this post. TrackBack URI

Leave a comment

© 2019 Deborah's Developer MindScape   Provided by WPMU DEV -The WordPress Experts   Hosted by Microsoft MVPs