Creating timestamps with sub-second accuracy

Author: Atle Borsholm

The TIMESTAMP function in IDL is good when you need to write log files and chronologically sort entries from multiple files. However, a common problem is that the number of digits is not enough when code runs quickly. This alternative routine shows how to improve timestamps into fractions of seconds. On Windows,"systime" is normally limited to about 1ms. But on Linux the accuracy can be closer to 1 microsecond.


Here is a comparison of the standard TIMESTAMP function and the output from the improved accuracy TIMESTAMP2 function.

ENVI> print,(t=timestamp()) & for i=0, 299 do if t ne timestamp() then print, (t=timestamp())


ENVI> print,(t=timestamp2()) & for i=0, 299 do if t ne timestamp2() then print, (t=timestamp2())












For TIMESTAMP2 the update rate is about 1ms running on a Windows 7 OS in this case. The following is a listing of the source code for the TIMESTAMP2 function:

function timestamp2

 compile_opt idl2,logical_predicate

 t = systime(1)

 str = systime(0, floor(t), /utc)

 td = 0d

 reads, str, td, format='(c())'

 td += (t-floor(t))/(24*3600d)

 tstr = string(td, format='(c(cyi4.4,"-",cmoi2.2,"-",cdi2.2,"T",cHi2.2,":",cmi2.2,":",csf06.3))')

 return, tstr




Comments (0) Number of views (89) Article rating: No rating

Categories: IDL Data Point





Guest Request - How to create an animated plot video

Author: Jim Pendleton

Very recently I had the opportunity to go for a hike here in Boulder with a friend from my college years.  He's now a successful scientist, using IDL since about 1985.  I sense causality.

Of course, since we're both science and software geeks, the topic of conversation rapidly devolved from family and the rejuvenating power of nature, to general health topics for 40-somethings, and coding quandaries.

He had a practical question for me.  "Let's say I want to animate a plot and programmatically generate a video for a presentation.  Is this easy to do in IDL these days?"

He'd not studied the What's New sections of the IDL documentation for a couple releases.  Science takes precedence, and his impressions were based on the now-obsolete MPEG_WRITE functionality.

The answer of course is that IDL now makes this task very simple, indeed. Here is a quick example, animating a simple sine wave plot.

(Recall that as of 8.3, you can copy and paste lines including the IDL prompt from this web page to the Workbench IDL Console, and the "IDL>" string will be stripped automatically.)

First, generate your wave form using the PLOT function.

IDL> x = findgen(361)*!dtor
IDL> y = sin(x)
IDL> p = plot(x, y)

For this exercise, we'll generate an output MPEG-4 file in your temporary directory.

IDL> file = filepath('test.mp4', /tmp)

We'll use the Plot::CopyWindow method to scrape the bitmap from the plot graphic.  This is returned in exactly the shape the WRITE_VIDEO procedure requires as input.

If you are generating output using Direct Graphics, the equivalent action is to call the function TVRD(TRUE=1) .  And if you are using Object Graphics, get the IMAGE_DATA property from your destination object, such as an IDLgrWindow.

IDL> write_video, file, p.copywindow(/antialias), handle = handle, $
IDL>        format = 'mp4', video_fps = 30

We've now written the first frame of our video.

Here, we've specified the format explicitly as MPEG-4 and the playback rate for the video as 30 frames per second.  On the first call to WRITE_VIDEO the HANDLE keyword is an output.  We will use it as input when writing the remainder of our frames.

Let's loop over 360 degrees by a single degree and update the plot at each step.  Our animation will produce about 12 seconds of video in playback.  (361 frames/30 frames per second.)

IDL> for i = 0, 359 do begin & $
IDL>    y = shift(y, 1) & $
IDL>    p.setdata, x, y & $
IDL>    write_video, file, p.copywindow(/antialias), handle = handle & $
IDL> endfor

Once we have written all our frames, explicitly close the video file to commit the changes.

IDL> write_video, /close, handle = handle

If you're running an OS that understands file associations, you should be able to simply SPAWN the file from IDL to verify its playback.

IDL> spawn, file, /hide

That was indeed easy.

If you want to "slow down" your animation at a critical point for highlighting, simply write a single image buffer multiple times.  For example, you might add this code snippet to slow down the animation during playback's central frames:

    if (i gt 90 && i lt 270) then begin
        for j = 0, 3 do begin
            write_video, file, p.copywindow(/antialias), handle = handle

The WRITE_VIDEO procedure is a wrapper for the IDLffVideoWrite class which in turn wraps the popular open-source FFmpeg library.  WRITE_VIDEO provides options for writing an audio stream to your video as well.

In the online help, see the note on "Replacing the FFmpeg Version" in the Creating Video topic if you need to write your animations in a format whose support is outside the defaults in IDL, such as H.264 video format or MP3 audio.

Comments (0) Number of views (250) Article rating: No rating

Categories: IDL Data Point





Basic information about using IDL on Linux

Author: David Starbuck

Recently, a user contacted us because they were transitioning from using IDL/ENVI on Windows to Linux. The user was very familiar with IDL on Windows but had never used it on Linux, and they were wondering if we had any articles or documentation help them. To help them, I put together the following list of information that I thought might be useful to users who are new to using IDL on Linux:

0) Installing IDL

If IDL is not installed on the system, the first thing you need to do is download the Linux version of IDL from our website and follow the installation instructions to install it.


The directory path "/usr/local/exelis" is the typical/default location where IDL is installed.  The installer for IDL will ask if you want to setup symbolic lines. If you answer"yes", then the IDL environment will be permanently set up on your system. If you answer "no", then you will need to set up the IDL environment every time you launch IDL.

1) Launching command-line IDL

If symbolic links were setup when performing the IDL installation, you should be able to launch IDL by simply entering the command into the terminal:


However, if these links were not setup, then you will need to setup the IDL environment before launching IDL. To do this, you need to source the correct file depending on which shell you are using:


$ source /usr/local/exelis/idl83/bin/idl_setup.bash
$ idl


$ source /usr/local/exelis/idl83/bin/idl_setup
$ idl

You can determine which shell you are using by entering the following command:

echo $SHELL

If you are successful in launching IDL, it will bring up and IDL prompt where you can compile and execute your IDL Programs:

IDL Version 8.3 (linux x86_64 m64). (c) 2013, ExelisVisual Information Solutions, Inc.
Installation number: -99999999999999
Licensed for use by: Exelis VIS IDL floating licenses


If you are familiar with the IDL executive commands, they may be useful in this environment:


2) Launching ENVI and using a headless system

You can launch ENVI from IDL using the following command:

IDL> en=envi()

If you are using a headless system (a system without a GUI that cannot display images), then you will want to use the HEADLESS keyword when launching ENVI:

IDL> e=envi(/headless)

In addition, if you are on a headless system, use the Z-buffer if generating Direct Graphics with IDL:


Also, on a headless system, use the BUFFER keyword when using the (New) Graphic. For example:

p = plot(findgen(10), /buffer)
p.save, "plot.png"

3) Differences between IDL programs on Linux and Windows

IDL programs are multi-platform. Therefore, for the most part, you should be able to run the same IDL programs you ran on Windows without changes. However, there are few differences between IDL on Linux and Windows.

One of the  main differences between IDL on Linux and Windows, is that the graphics device on Windows is the "WIN" device, and on Linux, it is the "X" device. Therefore, if you want to display something using Direct Graphics, you need to use the "X" device instead of the "WIN" device:


In addition, if you want to change the graphics preferences you will need to change the X graphics preferences:


Another minor change you might need to consider is that on linux directories are separated using the "/" character instead ofthe "\" character used by Windows.

4) Editing programs

If you need to edit programs on your Linux system, then there are a few options you can use to do this:

A) idlde

If you are using a headless system, then you can launch the IDL Development Environment and use it the same way you use it on Windows.You can launch it by typing "idlde" instead of "idl" into the terminal.

B) Use another text editor

If you cannot use the IDL Development Environment (ex: ifyou are on a  headless system), then youcan edit IDL programs using a standard linux text editor. A couple populareditors are EMACS and VI:



5) How to manage paths preferences

The best way to manage paths and preferences is to use IDL commands such as PREF_SET and PREF_GET. Some examples of how this can be done are shown on the following help page:


6) Linux Terminal Commands

If you have experience using the DOS command-line, you might find the Linux command-line environment to be pretty familiar. The following website provides a list of DOS commands and their equivalent in LINUX:


Some commands that I use a lot are listed below:

cd - Changes the directory location
ls -  List the contents of a directory
mkdir - Make a new directory
more - displays the contents of a file
pwd -  displays thecurrent directory location

Comments (0) Number of views (311) Article rating: No rating

Categories: IDL Data Point

Tags: Linux




Subsetting ENVIRasters for ENVITask

Author: Brian Griglak

A few months ago I wrote about ENVITask, the new API for analysis in ENVI 5.1.  Anyone who has looked at this API will notice that it the task objects do not contain parameters for spatial or spectral subsetting like ENVI_DOIT does with POS and DIMS.  Adding these parameters to every task causes unnecessary internal complication, and isn't needed due to our "raster is a raster" philosophy.  If you only want to run a task on a subset of the raster, it's up to you to perform the subsetting before invoking the task.  This is accomplished with the ENVIRaster::Subset() method, which returns a new ENVIRaster object that provides access to only that subset.

As the docs describe, there are three forms of subsetting that can be performed:

  • spatial subsetting using SUB_RECT
  • spectral subsetting using BANDS
  • pixel masking using ROI


You can use any or all of these keywords together, though if you specify by SUB_RECT and ROI the spatial subsetting is performed first, then the ROI masking is done.

Spatial subsetting is performed using the SUB_RECT keyword, which is expressed in pixel coordinates as an array [ leftCol, topRow, rightCol, bottomRow ].  These values will be clamped to the raster if you specify negative values or values too large.

Spectral subsetting is performed using the BANDS keyword, which is either a scalar band index or an array or band indices.  These indices are 0-based, and can't be repeated, but the order of them is significant and can be used for band reordering.  So setting BANDS=[0,1,2] will give you a different output raster than BANDS=[2,1,0].  Out of range band indices will throw an error.

Pixel masking using an ENVIROI is performed using the ROI keyword, which is set to the ENVIROI object reference.  This won't change the spatial or spectral extents of the raster, but it will mask out any pixels that aren't inside the ROI.  If you're writing your own extensions or processing scripts, you have to use the PIXELSTATE keyword when you call ENVIRaster::GetData() or ENVIRasterIterator::GetData().  As the documentation of these two methods explains, this keyword will be set to a byte array of the same dimensions as the data, which uses a bitmask to tell you whether each pixel is valid or not and why not.  For most purposes, the only question is whether the PIXELSTATE value is 0 or not, where 0 means valid and any non-zero value means invalid.

Here is an example that loads a Quickbird mosaic, chips out a small spatial subset with only the visible bands and calibrates it to a radiance image, and then loads the resulting output in a raster layer to compare to the original raster:

; load the raster

nv = ENVI()

inputFile = Dialog_Pickfile(TITLE='Select a file to calibrate')

oRaster = nv.OpenRaster(inputFile)


; display the raster in view

oView = nv.GetView()

oLayer1 = oView.CreateLayer(oRaster)


; subset the raster for task processing

subRect = [700, 900, 1000, 1300]

bands = [0, 1, 2]

oInput = oRaster.Subset(SUB_RECT=subRect, BANDS=bands)


; load the enviTask

oTask = enviTask('RadiometricCalibration')


; Set parameters

oTask.INPUT_RASTER = oInput

oTask.CALIBRATION_TYPE = 0 ; Radiance

oTask.OUTPUT_DATA_TYPE = 4 ; Float

; run the task


; display the results

oLayer2 = oView.CreateLayer(oTask.Output_Raster)


A screen shot of this code shows the results:

Comments (0) Number of views (312) Article rating: No rating

Categories: IDL Data Point





The Case For Static Methods

Author: Dain Cilke

A new feature introduced in IDL 8.3 is the ability to create static methods on IDL objects. For those new to the world of Object Orientated programming, static methods are essentially a normal IDL function, except the function is attached to an object.  Let's take a look at a simple object definition:

function STATICEXAMPLE::doit, x

  compile_opt idl2


  return, x/2




  compile_opt idl2


  self = { staticexample, $

           inherits IDL_OBJECT, $

           _count: 0b $  ; "unused" member variable



In this example, our object has a function DOIT which doesn't actually depend on any member variables or other functions defined by our object. Traditionally, to call this method you would have to do the following:

obj = StaticExample()


However, since DOIT doesn't actually depend on the state of the object or use methods defined on the object, it makes sense to mark DOIT as a static function to give it greater usability. We can do this by adding the static flag to the DOIT compile_opt:

function STATICEXAMPLE::doit, x

  compile_opt idl2, static

By marking the method as static, I can now reference the method in a single call:


Or if you have a STATICEXAMPLE object already:


Note: It is best to avoid calling static methods this way.  It is a lot easier to read your code if you don't have to double check every method to see if it is static or not.

" OK, but what benefit does marking a function as static give me?"

1.       The ability to group functionality into a single object. 

For example, say you have two sets of function to read and process different file formats: ff1_Read(), ff1_Process(), ff2_Read(), ff2_Process(). I can use static methods to group functionality into meaningful objects. In this case, I would choose to group my methods by functionality to create READ and PROCESS objects. So my calls would now be: Read.ff1(), Read.ff2(), Process.ff1(), Process.ff2().  This, in turn, leads to...

2.       Content assist! 

By converting your functions into static methods you can use content assist to help find that function call. Now, when working in the workbench, I can press CTRL+Space on my Read or Process objects and find all the file formats which I can open or process. 

3.       Avoid naming conflicts. 

When IDL compiles a function, it uses the code of the first function which matches the name of the compiled function. As such, if you have a function called Colortable and IDL releases a new function called Colortable, one set of functionality will be unaccessable depending on the ordering of your PATH (this has probably happened to you, and we are sorry for the confusion it causes). However, by wrapping your functionality under a static method, the chance of a namespace collision greatly reduces and leads us to...

4.       Libraries!

If you have a library (or even a bunch of useful functions) and you want to give to your coworker, with static methods you now have a dedicated object to contain all of your functions. So, instead of having an ugly tag hanging off the front of your functions (i.e. MyAwesomeCode_MyAwesomeFunction()) you can have a pretty object (i.e. MyAwesomeCode.MyAwesomeFunction()). Plus, it means your objects benefit from content assist, not having to worry about naming conflicts, pretty method names, and overall making your code more readable.



When using static methods there are a couple of things to be aware of. Static methods are not normal methods on objects.  They are not aware of any other methods or member variables.



Comments (0) Number of views (480) Article rating: No rating
12345678910 Last




© 2014 Exelis Visual Information Solutions