Skip to content

«
»
 

Optimized method to read CSV file in VEE

20 October 2009, 12:13 pm

Bookmark and Share

How to read data from CSV file in Agilent VEE? The code can be as complicated as shown in Figure 1 if we dont know VEE’s feature well. The code in Figure 1 is taken from a real application.

Read CSV file in Agilent VEE

Figure 1: Read CSV file in Agilent VEE

The pseudo code shown in Figure 1 is:

  • Open the file dialog, to let user select the CSV file to open
  • The ‘from file’ container open the csv data file, with the setting: “Read text x token exclude: “\n” Array:*; it means to read the csv file, then split the content into 1D array, with the “\n” (new line) as seperator
  • Remove the 1st row of the data, as the 1st row data is the heading/title
  • Use the ForEach loop, to process each row.
  • Use the FromString to split each row into columns, and collect the data into the container
  • save the data into a variable

The problem with this code are:

  • it create too many unnecessary containers, especially the ForEach loop. This is expensive solution in terms of performance.
  • The programmer actually forgot to close the file in the FromFile container. This is a bad practice.

Agilent VEE is design with optimize processing in array. The above code can be simplified and optimized to what shown in Figure 2:

Optimized and simplified way of reading CSV data in Agilent VEE

Figure 2: Optimized and simplified way of reading CSV data in Agilent VEE

The pseudo code for Figure 2 is:

  • Open the file with FromFile container. Read the csv file content into 2 dimensions array, by splitting it with “\n” and “,” deliminators. Close the file afterward.
  • Save the file into a Record variable.

With this solution, the optimized code perform much faster than original program. The 1st program takes 55.26ms for 10 rows of data. While the second approach use about 2.57ms, which is about 21.5 times of improvement. Also, it makes the code much cleaner and easier to read.

Refer to optimize_read_csv.zip file for detail. The UserFunction Analysis_ReadFromCSV shows the original implementation. I make the optimize method into a UserFunction: ReadCSV(FilePath, Number Of Column) which will return the csv data into a 2 dimension array.

«
»