SANE V2 - again...

abel deuring (a.deuring@satzbau-gmbh.de)
Sun, 29 Aug 1999 18:54:10 +0200

Hi all!

During the last weeks, some really interesting
discussions have started in this mailing list, especially
the one about interoperability of SANE and TWAIN, and the
one about the future of the SANE standard.

The latter became unfortunately quite personal, and some
mails were not very fair, in my opinion. The debate about
mime types versus more frame types has thereby reached a
dead end, and I don't want to make an immediate contribution
to this discussion. Instead, I'll try to go a step back to
discuss, which new demands appeared, and to outline how
these demands might be approached with Sane.

The debate about additional frame formats vs. mime types
arose from three points:

(a) that some high speed scanners and digital cameras
deliver their image data in compressed format.

(b) that some scanners deliver data in addition to the
usual RGB or gray scale data, like some slide scanners,
which produce infrared data, or high speed scanners, which
produce additional data in ASCII format. Another example
might be a digital camera being able to deliver the date and
time, when a photo was taken.

(c) if Sane should be extended to "completely different
data" like audio.

(b) and (c) are about data content, while (a) is about data
representation.

Regarding (c), I don't have any ideas and no emotions, and I
will therefore not make any further comments.

Please correct me, if there were more topics -- I was in
vacations, and I had not enough time to read all mails sent
this list during the last weeks very carefully.

Regarding data representation, my first thinking was that
the communication between a backend and a frontend becomes
unneccesarily complicated, if there is more than one
representation. On the other hand, it does not make not much
sense to receive compressed data from a scanner, let the
backend decompress this data, and let the frontend compress
the data again. Further, decompressing JPEG data and
compressing it again into this format might cause a loss of
information. Nevertheless I think that the Sane standard
should focus mainly on its current internal format
definitions. I will come back to the question of different
data formats vs. keeping the standard simple later.

The question how to handle data content "beyond RGB and gray
scale" is far more difficult. Any attempt to map these data
somehow into know formats seems to me questionable, even in
the comparatively simple case of an infrared channel of a
slide scanner: Of course it is possible to declare this
channel to be the alpha channel, and this will indeed
improve the image quality to some extent. But this is not a
proper solution: An alpha channel simply cuts out scratches
and dust from the image data. A proper solution would be,
that a function (or a special program) interprets the data
from the infrared channel as "replace the RGB data of the
pixels marked in the infrared channel as dust or scratches
by interpolating their neighboring 'valid' pixels". (A
short digression: Since slide scanners are quite common
meanwhile, it might be helpful to have a library function in
the Sane package which does the "dust removal". Such a
function is capable to convert the RGB/infrared data
properly into pure RGB data. But this does not need to be
covered by the Sane standard: Even the far more important
functions from sanei_scsi.c are not covered by it.)

Even more exotic situations have been mentioned during the
debate, like scanners delivering spectral data from IR to
UV. This data can of course not be represented as RGB -- but
it might be useful to convert thiy type of data into RGB for
a preview.

Another somewhat exotic example are some old drum scanners:
At least the combination of a Crosfield Magnascan scanner
and a Sun IPC box with special control software for this
scanner produces TIFF files with CMYK data. I don't know, if
the RGB -> CMYK transformation is done in the IPC box or in
the scanner, but since these scanners were some years ago
directly connected to imagesetters, I assume that the RGB ->
CMYK transformation is done by the scanner. While it is
mathematically quite easy to make again a CMYK -> RGB
transformation, this means a loss of information. A good
CMYK representation knows about specific details of the
printing technology: CMYK data for an ink jet printer can be
quite different from CMYK data for an offset printing press.

A more common situation might be a scanner / backend
combination which delivers colour calibrated data, generally
represented in the XYZ colour space or colour spaces derived
from XYZ, like CIELab. While up to now nobody in the Sane
community has cared about colour calibration, this could
become an issue, if Sane is implemented for Mac OS X: Since
Mac OS X is basically a Unix, it should be not too difficult
to get Sane running. On the other hand, the prepress
industry, the stronghold of Mac computers, is increasingly
using colour calibration, so that Sane would be not that
useful for many Mac users, if an integration of colour
calibration into Sane turns out to be quite difficult. More
informations on colour calibration can be found at
http:://www.color.org (especially about the ICC standard),
and at http://www.fogra.org (unfortunately, the latter site
contains much of its informations only in German).

So far about requirements on new or additional data
representations and contents. Now, what is the attraction of
the Sane standard and of the Sane software package? For me,
as a person working on a backend for an "ordinary" flatbed
scanner, there are three points:

- The "OS abstraction layer" sanei_scsi.[ch] enables the
backend to run on a number of different Unixes.

- The API defined by the Sane standard is simple in the
sense the I do not have to deal with a user interface (with
the exception of declaring a bunch of options);

- but it is nevertheless quite powerful in that a backend
can declare a wide variety of options, so that the
capabilities of a scanner can be easily presented to a
frontend and a user.

n interesting detail about the last point is that the
frontendA generally does not know anything about the meaning
of the options. (The "well known options" like tl-x are of
course an exception.)

A similar viewpoint could be taken on the scan data: At
least a generic frontend like xscanimage or scanimage does
not need to know anything about the data content: It's task
is mainly to control the scanner and to feed the scan data
to another program or into a file. (Oliver's xsane is
somewhat different, since it is able to perform
manipulations on the scan data, like gamma correction.)
Regarding scanner control, a preview should of course be
possible, if this makes sense for the particular device, and
in this case a backend should be able to deliver preview
data in a "well known format" like SANE_FRAME_RGB or
SANE_FRAME_GRAY. Therefore, my suggestion is to split the
requirements for the scan data format and scan data content
into two parts:

- The preview data may be only in SANE_FRAME_GRAY,
SANE_FRAME_RGB, SANE_FRAME_RED, ...GREEN, ...BLUE. If
necessary, the backend must transform some other internal
data content or data representation into one of these
formats. (For "better known" data representations like
xyz-compression and "better known" data contents like RGB +
"infrared for dust removal" library functions might be
developed which transform the scan data into RGB or gray
scale.

- The "real" scan data may be either in the same format as
the preview data (if this is reasonable for the particular
device), or in a format chosen by the backend. In the latter
case, the backend must support the frontend by giving hints
how to handle the data. At present, I don't know if these
hints should have the form of MIME types, as Andy suggested,
or if there might be better ways. Probably the most complex
question is how to handle several sets of different data for
one scan, like the additional bar code data produced by
these high speed scanners: Should all these data be
considered as one data stream by the backend / frontend API,
or should there be provisions to allow multiple data
streams? I am sure that Tom Martone and other people working
with this class of scanners have already thought about it,
so I would like hear comments from them.

Finally, I think that a major revision of Sane should
include language support. I know some users who would really
appreciate it, if it would be possible to display e.g. the
resolution option as "Auflösung".

A very rough, incomplete and perhaps too simple suggestion
for two library functions:

Instead of simply displaying a variable of the type
SANE_String, a frontend should call a function

SANE_String* sane_translate_from_english(const SANE_String* text,
const SANE_String* language)

and display the result of this function.

When handing a SANE_String value to the backend, a frontend
should call

SANE_String* sane_translate_into_english(const SANE_String* text,
const SANE_String* language)

To avoid failures of shell scripts like "xerox", the latter
function must of course be able to accept English texts as
input without trying to translate them.

Abel

--
Source code, list archive, and docs: http://www.mostang.com/sane/
To unsubscribe: echo unsubscribe sane-devel | mail majordomo@mostang.com