Is IDL 8.5 able to read tables with strings, dates and values in it?

Question

I am very new in the IDL scene and I am struggling for hours with a problem and I hope you can help me:

So right now I am trying to read data from a table ("file.txt"). I would like, that every column is saved in a variable (I thought about using STRARR)

I found this tutorial: http://www.idlcoyote.com/tips/ascii_column_data.html

This is very useful, when you want to read numbers for every column, which works fine. This is the table from the tutorial above:

Tutorial table:

Experiment 01-14-97-2b9c
No. of Data Rows: 5
Temperature         Pressure          Relative Humidity
  20.43             0.1654                 0.243
  16.48             0.2398                 0.254
  17.21             0.3985                 0.265
  18.40             0.1852                 0.236
  21.39             0.2998                 0.293

Code:

OPENR, lun, "tutorial.txt", /GET_LUN
header = STRARR(3)
READF, lun, header

data = FLTARR(3, 5)
READF, lun, data
temperature = data(0,*)

print, data
print, temperature

Output data:

  20.4300     0.165400     0.243000
  16.4800     0.239800     0.254000
  17.2100     0.398500     0.265000
  18.4000     0.185200     0.236000
  21.3900     0.299800     0.293000

Output temperature:

  20.4300
  16.4800
  17.2100
  18.4000
  21.3900

---

Looks quite good, for numbers. But what about when I have strings with dates, times, but also numbers in it, like this:

My table:

Experiment 01-14-97-2b9c
No. of Data Rows: 5
Date              Start time             End time             Value
12-Feb-2002       05:08:10               06:08:30             20
08-Mar-2002       07:35:38               09:25:59             100
20-Jun-2002       12:30:35               16:15:18             5536
25-Jul-2002       04:02:06               07:02:58             5822
02-Aug-2002       23:30:25               23:55:22             456

The code above won't work anymore. When I am using this my_var= data(0,*), the whole data will be saved in variable my_var, of course because the data are no more looking as columns, but in a whole row.

FLTARR is setting this data

12-Feb-2002       05:08:10               06:08:30             20

to this result (of course because of FLTarr)

12.0000
5.00000
6.00000
20.0000

And STRARR is saving the data good in my_var, but without separating the columns.

What I want:

I would like to have every column in one variable, so that I can handle these variable data later in another code.

dates = data(0,*)
starts = data(1,*)
ends = data(2,*)
values = data(3,*)

print, starts

Output: Start time

05:08:10
07:35:38
12:30:35
04:02:06
23:30:25

(and also the rest of my variables)

I hope you can help here. Maybe I misunderstood something, if so please let me know it.

For any other suggestion or solution I would be grateful.

Thanks in advance!

Answer 1

My suggestion would be to use STRSPLIT in some manner, either each line as you read it or all at the end.

Here is an example of doing it all at the end. First, read the data into a data array (ignoring the header array):

IDL> openr, lun, 'file.txt', /get_lun
IDL> header = strarr(3)
IDL> readf, lun, header
IDL> data = strarr(5)
IDL> readf, lun, data
IDL> free_lun, lun

Then split on whitespace:

IDL> tokens = strsplit(data, /extract)

And, finally, extract elements by position:

IDL> dates = (tokens.map(lambda(x: x[0]))).toarray()
IDL> starts = (tokens.map(lambda(x: x[1]))).toarray()
IDL> ends = (tokens.map(lambda(x: x[2]))).toarray()
IDL> values = (tokens.map(lambda(x: long(x[3])))).toarray()

You've got your values now:

IDL> help, dates, starts, ends, values
DATES           STRING    = Array[5]
STARTS          STRING    = Array[5]
ENDS            STRING    = Array[5]
VALUES          LONG      = Array[5]
IDL> print, values
          20         100        5536        5822         456

UPDATE: Make sure you have done

IDL> compile_opt strictarr

in scope before these commands.

UPDATE: To sort these arrays by value:

IDL> ind = sort(values)
IDL> values = values[ind]
IDL> dates = dates[ind]
IDL> starts = starts[ind]
IDL> ends = ends[ind]