>>>>> David Mosberger-Tang writes:
[...]
Mathias> swapspace) and I think not many people on Linux have. David
Mathias> mentioned that a temporary file wouldn`t be a nice solution.
Mathias> Here is my question: Why not ? Shouldn't it be possible to
Mathias> write three temporary files for the red, green and blue
Mathias> frame (of course not as image but the red bytes in a
Mathias> line...) so that each will consume 1/3 of the entire image
Mathias> in space? After the last pass the picture can be put
Mathias> together from those three files and send to stdout or file.
DM> It's certainly possible. Michael suggested using mmap'ing to
DM> provide backing store in the filesystem instead of the
DM> swap-space. Of the two solutions, I think mmap is technically
DM> preferable when there is plenty memory available. On the other
DM> hand, mmap is still quite a portability nightmare, so using
DM> regular files might still be better, overall.
Why not write to a single file (every third byte, for example), and on
each other pass, read a bufferful from the file, add in the bytes from
the scanner, and write the bufferful back to the file. This would be
a close alternative to mmap (which you could still use on systems that
have it implemented correctly).
In combination with the ``reserve space for the header'' idea I
mentioned before, this would never use more space than the resulting
image file.
Whenever separate files are used, you'll at least double the size of
the space required.
DM> However, the real issue is: what if you have tons of
DM> memory/swap-space but not much free diskspace? Don't say that
DM> doesn't happen.
Okay, I won't. ;)
Seriously, with the above scheme (either mmapping or writing to a
single file and using a separate in-memory buffer), the question of
disk space is moot. If anything, your large memory should be used by
the kernel to help cache this file and save the price of disk i/o.
DM> To me, it seems like we should use memory/swap-space when it
DM> looks like the image might fit in the available memory/swap-space
DM> and use temporary files otherwise. E.g., on Linux, we could base
DM> this decision on MemTotal: as reported in /proc/meminfo.
I dislike this sort of decision making... by Murphy's law, it will
make bad decisions in at least a few circumstances. By using a
constant in-memory buffer, the total space requirement will never
exceed imagesize + reserved_header + buffersize. Otherwise, we're
talking at least 2 * imagesize.
You have to write to disk *sometime*, and if it doesn't take any
additional space, it might as well be during scanning.
FWIW, I'm a big fan of doing things in-place rather than copying back
and forth. Memory and disk caching mechanisms are designed to handle
this behaviour, so it might even be a performance gain, too.
Heck, I'll even help with implementation if that's the extra push
that's needed to convince any skeptics. :)
-- Gord Matzigkeit | Proudly running pieces of the GNU operating system. gord@m-tech.ab.ca | Jacques Cousteau loved programming in assembler.
-- Source code, list archive, and docs: http://www.azstarnet.com/~axplinux/sane/ To unsubscribe: mail -s unsubscribe sane-devel-request@listserv.azstarnet.com