July 11, 2019

IMCUR - the image cursor

The current oven menu GUI supports interaction with the image cursur via two keycodes. Typing "i" yields IMCUR behavior and typing "o" yields IMCURG behavior. The later is a "goto" based on the value returned by the image cursor. What happens is that a tiny "donut" like cursor appears on the image display. You move this to a location of interest and type return and information gets returned to the GUI where it is interpreted in some fashion.

Part of this interpretation is done by a routine coded in SPP that converts image coordinates into oven coordinates. This is (was) done by a call to c_timcur, with the comment "this translates the image cursor to oven RTZ coordinates". Oddly enough, this routine seems to be in procedure timcur which is in "ograph/txyimcur.x".

The mechanism for this is hidden in the bowels of IRAF, you can learn some things about it by reading (for example) the postscript renderings of the documentation for ximtool, which can be found in the x11iraf sources at "/u1/iraf/x11iraf_src/doc".

Ports and such

No man page, but you can rummage around and find old postscript documents, which seem pretty well done for their day.

As it turns out, ximtool listens on 3 "gizmos", as follows:

Whenever you start ximtool these days, you get the following message:
Warning: cannot open /dev/imt1o
This is harmless, but annoying. Another bizarre anachronism given that you have a better chance of getting hit by lightning than finding these device files (for named pipes) on your system. If you get sufficiently annoyed, you can tell ximtool not to try to listen on these named pipes (there are two actually, imt1o and imt1i for output and input, since them old pipes are unidirectional). Invoking ximtool as follows will get rid of the message:
ximtool -xrm "*input_fifo:none"
But, who in the world is going to type all that in? The warning message is less annoying. It might be possible to put some setting into the .Xdefaults file, but this is always highly error prone, with a confusing and ambiguous syntax (that happily is almost never used these days). I gave this a try and ultimately gave up. It would be nice if it could be made to work.
ximtool -inet_only
This tells ximtool to start up using only a TCP socket (the default is 5137). To specify a different port use:
ximtool -inet_only -port 5555
The question then becomes how in IRAF do you direct output to this "rogue" ximtool running on port 5555?

The following little nugget has more vital and important information about Ximtool (and xgterm) than all other resources combined!

It even has the secret incantation to place in the .Xdefaults file. Remember to use xrdb < .Xdefaulsts after editing it. This excellent document even includes that important tip.
XImtool*port:		5555
XImtool*input_fifo:	none
Also, you can redirect output in IRAF via the following:
setenv IMTDEV		inet:5137:foo.bar.edu
export IMTDEV = inet:5555:localhost
The first line is given as an example to send a display to a remote host (using the old csh syntax). I speculate that the second would work for my "rogue" ximtool on the local machine. And it does! (see below).

Here is an old (1996) question and discussion about talking to ximtool. It underscores the lack of protocol documentation.

Those UNIX domain sockets

So, how would an ximtool client connect to one of these?
[tom@trona ~]$ netstat -lp | grep imtool
tcp        0      0 0.0.0.0:ctsd            0.0.0.0:*               LISTEN      5030/ximtool
unix  2      [ ACC ]     STREAM     LISTENING     1639010  5030/ximtool         /tmp/.IMT1004
unix  2      [ ACC ]     STREAM     LISTENING     1639013  5030/ximtool         /tmp/.ISM1004
unix  2      [ ACC ]     STREAM     LISTENING     1639016  5030/ximtool         /tmp/.ISM1004_0
-- restart ximtool
[root@trona tom]# netstat -lp | grep imtool
tcp        0      0 0.0.0.0:ctsd            0.0.0.0:*               LISTEN      12525/ximtool
unix  2      [ ACC ]     STREAM     LISTENING     1760170  12525/ximtool        /tmp/.IMT1004
unix  2      [ ACC ]     STREAM     LISTENING     1760173  12525/ximtool        /tmp/.ISM1004
unix  2      [ ACC ]     STREAM     LISTENING     1760176  12525/ximtool        /tmp/.ISM1004_0
My uid is 1004, which explains part of what we see above. And the following tells us where 12525 comes from in the above, it is the PID of the ximtool process.
[root@trona tom]# ps -alx | grep imtool
0  1004 12525  4798  20   0   9340  7484 compat S+   pts/1      0:00 ximtool
0     0 12648 12480  20   0 215740   896 -      S+   pts/3      0:00 grep --color=auto imtool

[root@trona tom]# lsof -p 12525
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF     NODE NAME
ximtool 12525  tom  cwd    DIR              253,2    20480  2883585 /home/tom
ximtool 12525  tom  rtd    DIR              253,0     4096        2 /
ximtool 12525  tom  txt    REG               8,19  6270887 95177427 /u1/iraf/iraf_2.16.1/vendor/x11iraf/bin.linux/ximtool
ximtool 12525  tom    0u   CHR              136,1      0t0        4 /dev/pts/1
ximtool 12525  tom    1u   CHR              136,1      0t0        4 /dev/pts/1
ximtool 12525  tom    2u   CHR              136,1      0t0        4 /dev/pts/1
ximtool 12525  tom    3u  unix 0x0000000016390cc5      0t0  1756759 type=STREAM
ximtool 12525  tom    4u  unix 0x0000000019839300      0t0  1756760 type=STREAM
ximtool 12525  tom    5u  unix 0x00000000fc4b8a3a      0t0  1760170 /tmp/.IMT1004 type=STREAM
ximtool 12525  tom    6u  IPv4            1760172      0t0      TCP *:ctsd (LISTEN)
ximtool 12525  tom    7u  unix 0x00000000b42b522e      0t0  1760173 /tmp/.ISM1004 type=STREAM
ximtool 12525  tom    8u  unix 0x0000000049f28ae2      0t0  1760178 /tmp/.ISM1004_0 type=STREAM
ximtool 12525  tom    9u  unix 0x00000000b24c954f      0t0  1760176 /tmp/.ISM1004_0 type=STREAM
Note also that "ctsd" is a bogus translation of port 5137 based on an entry in /etc/services, as follows:
ctsd            5137/tcp                # MyCTS server port

In the ximtool source, the path for the address that gets bound to the unix domain socket is generated via the following.

sprintf (path, xim->unixaddr, getuid());
Apparently the usual value for "unixaddr" is "/tmp/.IMT%d", but this can be changed by a command line switch or X resource setting.

The mechanism

It all traces back to an ancient piece of hardware called the IIS. This is so old that even the retrocomputing gang on the internet don't know about it. Worse yet, Microsoft has some kind of web server called IIS -- and any search for IIS gets swamped by hits on this.

The IIS is (was) related to (I think was supplanted by) the VICOM. The VICOM is a similar piece of antique hardware. I think it had a 512 by 512 display and supported the IIS protocol (whatever it may be) over a serial port. For better or worse (mostly worse due to the lack of documentation) the ancient IRAF utilities ximtool, saoimage, and even the more recent ds9 all seem to support this crusty old protocol, but over a TCP connection (or a named pipe or fifo or such). Say what you will about it (and I certainly have) it still seems to work, once you pull the right set of black boxes out of the bag and dust them off.

Experimenting

I discovered the following more or less by accident. If I start up ximtool, and also get the CL running in some window, I can type:
print imcur
11.985 13.000 101 i
The way this works is that the "print" statement causes the donut cursor to appear in the ximtool window. I can then move the donut around using the mouse. When I type any key, I get the text output shown. I thought I was going to have to write an SPP program to experiment with this, but this makes it easier. What are the 4 values returned? The first two are easy, they are positions on the image with the origin at the upper left corner. The first value increases to the right, the second value increases downward. Both are in the range of a 512x512 image, presumably 0-511. The third value (101) is mysterious, but always 101 so far. The last value is the key typed. If you type return you get \015, if you hit the space bar you get \040. Apparently these values are octal (how quaint!). Clicking the mouse does nothing whatsoever, which is fairly annoying.

Running wireshark looking at port 5137 yields nothing. So I start ximtool using "ximtool -inet_only" and now when I type "print imcur" I get the following message. Apparently IRAF is using the named pipe method rather than a TCP connection to talk to ximtool.

vocl> print imcur
ERROR: Cannot open device (node!imtool,,512,512)
The following is interesting, but not very informative:
vocl> print envget("stdimage")
imt512
typing "gdevices" gives a long list of different frame buffers, but with no indication of how to change the protocol or port number to talk to them. Refering to the ximtool secrets link above, it suggests setting the IMTDEV environment variable (apparently this is not set by default, but if you set it before starting the CL you may get lucky).
setenv IMTDEV		inet:5137:foo.bar.edu
setenv IMTDEV		inet:5137:localhost
export IMTDEV=inet:5137:localhost
The first two lines pertain to the ancient csh, but any sane person these days is using bash, so the last line applies. And this works! I can now run ximtool -net_only and get a graphics cursor interaction. Now we can try wireshark. I fire it up, set a capture filter of "tcp port 5137" and tell it to capture on all interfaces. And I get action on 127.0.0.1 !! Wireshark is always tricky to use if you don't use it regularly. To save a capture to a text file, you use "Export packet dissections" which offers the option to save to a text file.

Here is what I see.

After the 3 way handshake to establish the TCP connection, there are 3 transmissions to port 5137 as follows:

16 bytes: 00 01 fd ff 0c 00 f6 7e 00 80 00 00 00 00 00 00
 6 bytes: 00 00 00 00 00 00
16 bytes: 00 80 60 ff 10 80 8e 00 00 00 00 00 01 00 00 00
After this, the donut appears on the image screen. Once a key is typed, ximtool responds with 160 bytes as follows:
0000  20 20 20 32 34 34 2e 39 39 32 20 20 20 20 33 32      244.992    32
0010  36 2e 30 31 30 20 34 30 31 20 78 20 0a 00 00 00   6.010 401 x ....
0020  0a 00 00 00 00 00 00 00 18 ad c3 ff 00 00 00 00   ................
0030  40 af c3 ff 10 ad c3 ff 18 ad c3 ff 50 10 09 08   @...........P...
0040  64 aa 3f 22 48 af c3 ff 58 ad c3 ff a3 70 1f 08   d.?"H...X....p..
0050  04 00 00 00 48 ad c3 ff 28 ad c3 ff 00 00 00 00   ....H...(.......
0060  00 00 00 00 00 00 00 00 30 ae c3 ff 00 00 00 00   ........0.......
0070  05 00 00 00 99 99 99 19 98 af c3 ff 7f 8a 1f 08   ................
0080  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0090  be ae c3 ff 01 00 00 00 00 00 00 00 06 00 00 00   ................
After sending an ACK for this data, the requestor closes the connection.

What could be simpler than that?! No telling what all that 160 bytes is all about, but what we want is verbatim in the first 32 bytes.

Look at the ximtool source

The first item of interest is this structure (which is 16 bytes in size).
struct  iism70 {
        short   tid;
        short   thingct;
        short   subunit;
        short   checksum;
        short   x, y, z;
        short   t;
};
The routine xim_iisio() seems to get called as a handler for any input, and the first thing it does is to read the above, which it calls the "IIS header". The heart of this routine is a big switch on iis.subunit & 077 The case for IMCURSOR (which is 020) seems to be what we want. Octal 020 is hex 0x10. Looking at the second 16 byte packet, we see "10 80" for "subunit". Byte swapping this and masking out the low 6 bytes, this indeed is the IMCURSOR value.

But what is the switch value for the first 16 byte packet? Here subunit is "0c 00" and if we byte swap this and mask 6 bits we get 0x0C. As near as I can tell, this value is not defined, so it will be ignored, and the following 6 bytes will be skipped.

I am heartened to see the following define:

#define SZ_IMCURVAL     160

The first word (tid) has a flag bit defined as 0x8000, which indicates a READ (otherwise a write is being done). Note in the above that the second 16 bit command has this bit set (so the cursor is being read). The first command does not set this bit, but does set 0x0100 -- no telling what this bit signifies.

The "thingct" value yields the number of bytes following the header on a write. For our first header, this value is "fd ff". We byte swap this to get fffd, then the negative of this (namely 3) is multiplied by 2 (since the PACKED flag is not set), yielding 6, just as it should.

Looking at the source, it is possible to turn on IIS debugging as follows:

[tom@trona ~]$ export DEBUG_IIS=9
[tom@trona ~]$ ximtool &
subunit=000014 tid=000400 nbytes=      6 x=100000 y=000000 z=000000 t=000000
subunit=000020 tid=37777700000 nbytes=    320 x=000000 y=000000 z=000001 t=000000
read cursor position
curval:    172.990    293.009 101 x
I don't see the expected announcement that the 6 bytes are being skipped, so I am not entirely certain of my analysis. Also note the nbytes=320. My guess is that the IRAF code wrongly just set 160 in the count field not realizing this would be doubled on receipt -- but apparently this is ignored on a READ anyway.

If we really want to understand all of what is in that 160 byte reply message we could dig deeper into the ximtool source (or perhaps even look at DS9). For now, everything I want is in the first 32 bytes.

Write a simple client

The idea here is simple. Write a network client and play back the magic bytes and see if I get the desired result. In fact I do and I see no compelling reason to understand the IIS protocol. In addition I find that it works just fine to skip sending the first 16 bytes along with the 6 zeros that go with it. Send the 16 magic bytes in the second message and bingo -- you get a 160 byte response with what you want in the first 32 bytes. Running this program, I get the following:
[tom@trona SPP]$ ./mycur
Newline at: 28
Result:    149.989    206.006 101 u

Have any comments? Questions? Drop me a line!

Tom's home page / tom@mmto.org