9

Apr

2015

Using EXIF Metadata to Show Photo Locations in Google Earth, via IDL

Author: Jim Pendleton

Have you ever needed an IDL API for retrieving a geographic location at which a digital camera photo was taken, one stored in a standard format such as a TIFF or JPEG?

On a lovely spring morning, I shot this photo of a flowering tree in full bloom near the Exelis building in Boulder, Colorado. When I grow older and daft-er, how will I remember where the photo was taken, plus or minus GPS accuracy?

 


IDL 8.4.1 includes a new READ_EXIF function for that purpose.


The input is the path to the image file, and the output is a HASH containing much useful metadata.


If GPS tagging was enabled on the device on which the photo was acquired, its contents will be stored in the inelegantly-named but standard key "ifd0_subifd0_GPS".


IDL> r = read_exif(dialog_pickfile())

IDL> if r.haskey('ifd0_subifd0_GPS') then GPSInfo = r['ifd0_subifd0_GPS']

IDL> print, GPSInfo, /implied

{

    "ifd0_subifd0_GPS": {

        "Exif_GPSInfo_GPSLongitude": [105.00000000000000, 14.000000000000000, 10.588900000000001],

        "Exif_GPSInfo_GPSAltitudeRef": 0,

        "Exif_GPSInfo_GPSLatitude": [40.000000000000000, 1.0000000000000000, 13.553400000000000],

        "Exif_GPSInfo_GPSAltitude": 0.00000000000000000,

        "Exif_GPSInfo_GPSTimeStamp": [14.000000000000000, 53.000000000000000, 34.000000000000000],

        "Exif_GPSInfo_GPSProcessingMethod": "ASCII",

        "Exif_GPSInfo_GPSVersionID": [2, 2, 0, 0],

        "Exif_GPSInfo_GPSDateStamp": "2015:04:06",

        "Exif_GPSInfo_GPSLatitudeRef": "N",

        "Exif_GPSInfo_GPSLongitudeRef": "W"

    }

}


The tag names are long and cumbersome, but they use a standard prefix.


IDL> pre = 'Exif_GPSInfo_GPS'


Let's show the location in Google Earth. First extract the longitude and latitude in decimal degrees.


IDL> Lon =Total(Float(GPSInfo[pre+'Longitude'])/[1.,60.,3600.])

IDL> Lon *= GPSInfo[pre+'LongitudeRef'].Compare('W') eq 0 ? -1 : 1

IDL> Lat =Total(Float(GPSInfo[pre+'Latitude'])/[1.,60.,3600.])

IDL> Lat *= GPSInfo[pre+'LatitudeRef'].Compare('N') eq 0 ? 1 : -1


Next, create some KML containing the placemarker location. 


IDL> l = '<?xml version="1.0" encoding="UTF-8"?>'

IDL> l += '<kml xmlns="http://www.opengis.net/kml/2.2">'

IDL> l += '<Placemark>'

IDL> l += '<name>Photo Location</name>'

IDL> l += '<description>Attached to ground.  Could also use GPS altitude.</description>'

IDL> l += '<Point>'

IDL> l += '<coordinates>' + StrJoin([Lon, Lat, 0], ',')  + '</coordinates>'

IDL> l += '</Point>'

IDL> l += '</Placemark>'

IDL> l += '</kml>'


Write the KML to a temporary file.


IDL> tempfile = filepath('test.kml', /tmp)

IDL> openw, lun, tempfile, /get_lun & printf, lun, l & free_lun, lun


If ".kml" is registered as a known extension on your computer, you should be able to simply SPAWN the path to the file.


IDL> spawn, tempfile, /hide, /nowait



Be aware that GPS tagging of photos is generally disabled by default on most mobile devices these days to support a greater level of privacy.  To toggle GPS tagging on, one generally needs to locate a "Settings" or "Options" menu associated with the camera application on a particular device.

Comments (0) Number of views (315) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

2

Apr

2015

User-Defined ENVITasks in ENVI 5.2 SP1

Author: Brian Griglak

ENVI 5.2 SP1 has just been released, so I'm back with another blog post about the new features we have added to the ENVITask API. We have added 24 new tasks, to take the total to 82, but the most exciting feature is that you our customers now have the tools to create your own ENVITasks. We have made good progress on exposing ENVI's analytic capabilities as tasks, but there are so many more of you than there are of us, with a myriad of ideas and algorithms that there's no way we could satisfy all your needs if we didn't make it easy for you to define your own tasks. Enter the new ENVITaskFromProcedure class, which allows you to wrap almost any IDL procedure as an ENVITask.

Before I go into detail on how to use ENVITaskFromProcedure,allow me to convince you why you would want to go through the minimal effort in creating a task for your given procedure.

First and foremost is the fact that once you have your ENVITask working in desktop ENVI, it is trivial to copy the files onto an ENVI Services Engine server and get your task working in the cloud. The ESE server doesn't care if an ENVITask is part of the ENVI installation or a custom task created by you, it will just work. 

Another valuable feature of ENVITask is the automatic parameter validation that it provides. You don't need to check the type and cardinality of every input to your procedure, you define that in the task template and the ENVITaskParameter class takes care of all that work for you. This may not help the existing code you wrap as much, but as you write new algorithms with the intent of wrapping them as ENVITasks, you'll be amazed how much shorter and cleaner your code will be. The ENVITask framework allows you to treat IDL like a strongly typed language, which frees you to focus on algorithm implementation details and not robustness in the face of bad inputs.

As I mentioned in my last ENVITask blog post, in ENVI 5.2 we introduced the dynamic UI for ENVITasks. So you don't need to write custom widget code for each algorithm, you can just invoke the ENVIUI::SelectTaskParameters() function to pop up a dialog to enter all the input parameters for your task.

Hopefully I've at least piqued your interest in looking at ENVITaskFromProcedure, the next obvious question is "what rules does my procedure need to follow to be wrapped as a task?".  There are a couple, but they aren't too burdensome, and there are easy ways to create a wrapper procedure to fix any violations. Here are the rules, and the wrapper workarounds:

1. Only keywords are allowed, no positional parameters.
Keywords enforce a more explicit mapping between input values and intention.  If we had written the Gram-Schmidt Pan Sharpening task to have a single input parameter that was a 2-element array of ENVIRaster objects, then you need to read the documentation to figure out if it [ Pan, MSI ] or [ MSI, Pan ].  By having two separate scalar parameters named INPUT_LOW_RESOLUTION_RASTER and INPUT_HIGH_RESOLUTION_RASTER, the task is self-documenting and it's much harder to confuse which input is which.

If you have a procedure with positional parameters that you want to wrap as a task, all you need to do is create a new procedure that has only keywords, and then create a keyword for each positional parameter.  This wrapper procedure then invokes the original procedure, passing in the keywords in the appropriate order for the positional parameters.

2. The algorithm must be a procedure, not a function.
The return value of a function is a similar to the positional parameters, you can think of it as the parameter at index -1. We can't know what the type of this value is, or what you might want to name it, so we don't try. The solution to this is the same as the positional parameter case, create a wrapper procedure that calls the function and stores the function's return value in an output keyword.

3. The procedure must use ENVI5 API objects such as ENVIRaster, not ENVI Classic FIDs.
ENVI was around for quite a while before we released ENVI 5.0, and I'm sure there are many routines out there written using its API of FID, DIMS, POS.  This code doesn't have to be rewritten, it is easy to create a wrapper procedure that can translate an ENVIRaster input into the values needed for those keywords:
     fid = ENVIRasterToFid(inRaster)
     dims = [ -1, 0, inRaster.nColums-1, 0, inRaster.nRows-1 ]
     pos = Lindgen(inRaster.nBands)

That's pretty much it, if your procedure obeys these rules, or is wrapped by one that does, then you can wrap it in an ENVITask. As the help docs for Custom Tasks describes, you create your .task template file, make sure the"baseClass" attribute is set to "ENVITaskFromProcedure", and the "routine" attribute is set to your procedure name. A good starting point for this is the BandMathExample that we included in the ENVI 5.2 SP1 release. On Windows you can find it in the "C:\Program Files\Exelis\ENVI52\examples\tasks\bandmathexample" folder. This example includes the procedure beingwrapped, the .task template file, some PRO code to create a user extension forthe ENVI toolbox, and a PRO file to test the task.

Comments (0) Number of views (466) Article rating: 5.0

Categories: IDL Blog | IDL Data Point

Tags:

26

Mar

2015

The ESE API for IDL

Author: Doug Edmundson

With IDL 8.4.1, the ENVI Services Engine (ESE) has a PRO-code interface. This “ESE API” makes it easy for IDL programs to run ESE tasks, monitor them and get results. There are some other goodies in there as well.

First, let’s recall what ESE does. It is an application that provides on-demand IDL, ENVI and ENVI LiDAR processing as a web service  It can be run as an app or a service.  Client applications make requests that look like:

                 http://<host>:<port>/ese/<catalog>/<folders>/<service>/execute?<query-string>

A real world example for an ESE installation running on my CentOS machine, penny64, is:

                http://penny64:8181/ese/services/SyncService/ese_addition/execute?a=1&b=2

 The “ese_addition” task that ships with ESE simply adds two numbers. The result is in JSON form (edited for brevity):

        {

         "jobId": "42”,

         "jobStatus": "esriJobSucceeded",

         "jobStatusURL": "ese/jobs/42/status",

         "jobProgress": 100,

         "jobProgressMessage": "",

         "taskName": "/services/SyncService/ese_addition",

         "jobErrorMessage": "",

         "results": [

           {

             "name": "answer",

             "dataType": "double",

             "parameterType": "required",

             "value": 3

           }

         ],

         "messages": […]

       }

Any program that can make HTTP requests and parse JSON can use ESE to do processing.

Now, with IDL 8.4.1, the ESE API takes the complexity out of IDL programs invoking ESE tasks. The API lets you do the following:

 •             introspect an ESE installation with respect to services and their tasks

•             run tasks (synchronous and asynchronous)

•             monitor and manage job progress

•             retrieve job results

•             upload and download data

Any IDL program can use this API, be they desktop applications or tasks themselves. The API ships with IDL 8.4.1 and does not require a local ENVI nor ESE installation.

The ESE API consists of several classes that generally mirror ESE constructs. For example, ESE has the notion of services, tasks and jobs. So, the API has the ESEService, ESETask and ESEJob classes. There are also classes for ESE servers, folders and even task parameters. All this is in the IDL documentation, so let’s jump straight into an example. Let’s add two numbers:

       oTask = ESE.findTask( 'penny64', 'ese_addition' )

       oJob = oTask.run( a = 1, b = 2 )

       print, oJob.answer

Those three lines fetch the “ese_addition” task, run it and print the answer. The API takes care of all the details: creation of the HTTP request string, issuing the request, parsing the JSON result, error handling, etc. A less obvious duty of the API is to convert IDL variables to and from JSON (the transport mechanism). The API includes facilities for converting custom data types back and forth between IDL variables and JSON.

It should be mentioned that "a" and "b" are input parameters defined on the task and that "answer" is the name of the task's output parameter.

Now, let's try a more complex example that demonstrates task chaining, where the output of one task is used as the input of another.  Call it a workflow.  Let's add three numbers:

        oTask = ESE.findTask( 'penny64', 'ese_addition' )

       oJob1 = oTask.run( a = 1, b = 2 )

       oJob2 = oTask.run( a = oJob1.answer, b = 3 )

       print, oJob2.answer

There are a couple things to note:

 ·         the first job's output parameter, "answer", is just an IDL variable that can be used in the invocation of subsequent tasks (or anything else one wants to do in IDL)

·         one task can be run multiple times; each time, a brand new ESEJob object is returned

 Without going into all the details of the API, here are some of the more interesting features:

 •             run: Execute synchronous and asynchronous task in the style of IDL methods.

•             join: Wait for multiple jobs to complete before continuing.  Good for synchronizing jobs that are running in parallel.

•             monitor: Get notifications whenever a job's status changes.

•             conversion and validation: Program in the style of IDL without thinking too much about data type conversion and validation.

•             upload and download: Copy data between client and server.

 

Although the ESE API provides data "upload" and "download" functionality, the best practice is to keep the data where the processing is and not move it around. So, when the data is large, the mantra is to “bring processing to the data.”

As we’ve seen, the ESE API exposes the power of ESE task processing. Given the atomic nature of tasks, they lend themselves well to both linear chaining and parallel processing. The power of ESE tasks is now closer to your mind's eye.

Comments (0) Number of views (538) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

18

Mar

2015

What the *bleep* is IDL doing: COMPILE_OPT

Author: Dain Cilke

If you are a modern IDL programmer, the line compile_opt idl2 is at the top of every function/procedure/method/$MAIN program you write.  If it's not, I'm here to convince you it should be. 

First off, what exactly is a compile_opt. A compile_opt is a statement which is processed by the IDL compiler to change the behavior of the compiler.  It is processed in order of the statements (meaning if you put a compile_opt in the middle of your code, it's effect would only apply to code after the compile_opt).

What does compile_opt idl2 do?  The idl2 flag sets two flags for the IDL compiler, DEFINT32 and STRICTARRDEFINT32 sets the default size of IDL integers to be 32-bits rather than 16-bits.  This is useful because it aligns the default behavior of IDL with the standard size of a C language long, so you can rely on the more liberal -2147483648 and 2147483647 underflow/overflow values of type longSTRICTARR  prevents IDL from using parenthesis for array indexing.

Why is this important?  Consider the following:

       offset = input(0)

       data = input[100] * offset

What is input?  One of the difficulties of untyped languages is the ability to discern the intent of code from a small context.  In this case, input could either be: a function and an array, or an array which is being accessed two different ways (valid syntax without compile_opt idl2).  By always putting compile_opt idl2 at the beginning of every function, this ambiguity is removed.

You might be thinking to yourself, "But I like indexing array's with parenthesis, I'll just stick compile_opt idl2 in any new code I write".  In the words of a once great robot, "DANGER! DANGER, WILL ROBINSON!"  Consider the following:

test22.pro

PRO test22

  resolve_routine, 'a', /either,/compile_full_file

 

  print, 'Answer = ', b('stuff')

 

END

----

a.pro

FUNCTION b, istuff

  COMPILE_OPT idl2, hidden

 

  RETURN, 'b'

END

 

 

FUNCTION a, istuff

  COMPILE_OPT idl2

 

  RETURN, 'a'

END

If you try to run test22, the function b will not be resolved.  This is because the symbols for test22.pro are generated before the resolve_routine and since there is no reference to b it will be treated as a variable.  After the resolve_routine, the compiler will not override the variable type of b for the function.  This problem will also present itself if you sick compile_opt idl2 in the middle of your code.  When IDL gets confused, you lose.

Here are some simple tricks to follow to avoid these pitfalls:

  1. Always put compile_opt idl2 at the top of your function/procedure.
  2. If your function doesn't have compile_opt idl2, add it!  Worse case, you have to refactor array parenthesis to brackets.  But this simple refactor will reduce headaches down the road by making your code more readable and extensible.
  3. If you are working in legacy code (and insist it remain legacy code), be aware if external code is using compile_opt idl2. If it is, double check your function/procedure/variable names to prevent IDL from getting confused.

Comments (0) Number of views (656) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

12

Mar

2015

Logical vs. Bitwise Operators

Author: Benjamin Foreback

There are two different types of comparison operators in IDL: bitwise operators and logical operators.

A bitwise operator evaluates each bit of two expressions based on the logic that is defined by the operator. These include the AND, OR, NOT, and XOR operators. Each bit of the input expressions
will be compared independently of other bits. IDL will always evaluate both expressions. 

A logical operator is very similar to a bitwise operator in that it evaluates two conditions. IDL's logical operators are && (logical and), || (logical or), and ~ (logical not). There are two significant differences between a logical and bitwise operator. First, a logical operator will always return 1 (for true) or 0 (for false). Additionally, a logical operator will perform "short circuit" logic, meaning that if the outcome is known after only checking the first condition, the second condition is ignored. For example, when using the logical and, we know that both conditions must be true in order for the result to be true; if the first of the two evaluates to false, then the result will be false regardless of the second condition, and therefore the second condition will not be evaluated.

In addition to simply being a bit more efficient, the logical and is extremely useful when you first need to check for the existence of a value before checking a condition of it. For instance, I have a variable called var, and I want to write an IF statement that is executed if the variable is greater than 5. However, in my code, I don't know for sure that var exists, so first I must check N_ELEMENTS on it. With the logical and, this can be done on one line.


IF N_ELEMENTS(var) GT 0 && var GT 5 THEN BEGIN
   ; ...
ENDIF


If var doesn't exist in the code above, then IDL isn't going to check to see if var is greater than 5. If the bitwise AND operator was used, IDL would have thrown an undefined variable error when checking the second half of the if statement. To use AND, this logic would need to be on two different lines.


IF N_ELEMENTS(var) GT 0 THEN BEGIN
  IF var GT 5 THEN BEGIN
    ; ...
  ENDIF
ENDIF


So when would you want to use a bitwise operator? 

Well first of all, logical operators only work when the result from each expression is a scalar. For example, if I want to use a WHERE statement to check two different conditions on an array, then the bitwise operator needs to be used.


arr = INDGEN(6)
result = WHERE(arr GT 1 AND arr LT 4)


Substituting AND with && in the above expression would result in an error.

In less common cases, the conditional expressions might be the result of two different functions, and you want both functions to always be performed and not skipped (perhaps the functions do something to the variable as it's passed-by-reference, or maybe they change internal variables based on the input), then you could put them together with a bitwise operator.


IF my_function1(var) AND my_function2(var) THEN BEGIN
  ; ...
ENDIF


Note that if you wanted one to always run but didn't care about the second, you could use a logical operator and put the one you always want performed first.

You can even do tricks with bitwise logic. Because bitwise logic operates on all bits of each value and does so independently of all other bits, you get interesting results. For example:


PRINT, 3 AND 6


IDL will print 2. This is because 3 in binary is 011 and 6 in binary is 110. The result is... (first bit): 0 AND 1 = 0, (second bit): 1 AND 1 = 1, (third bit): 1 AND 0 = 0...  010, which is 2. This can sometimes be handy when processing image data to modify certain colors. A very useful example is that the logical NOT will convert a photograph to a negative.

As a final note, keep in mind that when multiple operations are performed in the same line of code, IDL will follow an operator precedence, similar to order of operations in mathematics.


Comments (0) Number of views (702) Article rating: 5.0

Categories: IDL Blog | IDL Data Point

Tags:

12345678910 Last

MOST POPULAR POSTS

AUTHORS

Authors

© 2015 Exelis Visual Information Solutions