PZ Myers. 2005 Apr 22. Image processing rules. <http://pharyngula.org/index/weblog/image_processing_rules/>. Accessed 2008 Nov 19.
Posted on M00o93H7pQ09L8X1t49cHY01Z5j4TT91fGfr on Friday, April 22, 2005
Image processing rules
I'm ahead of the curve. I've been teaching about image processing in biology for around 12 years now, and I've always told my students that there are some important rules about what you can ethically do to a scientific image. Nature has a short article on the subject this week, and they mention essentially the same rules—and I've been pushing the same stringencies in my classes.
Few journals have explicit policies, and of those that do, The Journal of Cell Biology has the most stringent guidelines. These allow alterations that are applied equally across an entire image, such as changes to contrast or brightness. They also permit some other corrections, such as adjusting the brightness of pixels in a certain range of colours — but only if details of the adjustment are spelled out. Changes to selected parts of an image, such as brightening one cell in an entire field or scrubbing out an ugly blemish, are prohibited.
They also cite an excellent article from the Journal of Cell Biology that spells out all the rules, which I'll definitely be handing out in my lab courses in the future. Just to summarize them briefly, though, here's what I consider important:
- Always archive the original image data. Only work on a copy.
- All enhancements must be global. You aren't allowed to select a region of the image and modify it alone, all operations must be applied equally to the whole image.
- The paintbrush, eraser, clone stamp, and other painting tools are absolutely forbidden.
- Document everything: optical settings on the microscope, scale, any modifications made.
- Don't copy and paste bits and pieces of images together to create composites that never existed. Photomontages of mult-field images are OK as long as it is clear how the pieces were spliced together.
The JCB article also gets into an odd argument about linear vs. nonlinear operation on pixel intensities that I don't find very important. It's fooling ourselves to think that optical properties are perfectly linear, that camera technologies are linear, or that printing output will be linear—we know they are not. I don't see a simple brightness adjustment using a constant as more virtuous than a histogram equalization. But their bottom line is not to oppose non-linear manipulations, but to insist on full disclosure, and I'm all for that.
Pearson H (2005) Image manipulation: CSI: cell biology. Nature 434:952-953.
Rossner M, Yamada KM (2004) What's in a picture? The temptation of image manipulation. J Cell Biol 166(1):11-15.
Posted by PZ Myers on 04/22 at 09:07 AM
Science • Research • 0 Trackbacks • Other weblogs • Permalink
Science • Research • 0 Trackbacks • Other weblogs • Permalink
-
Good point about linearity. It is not as though photographic film has a linear density response to illumination, and most of the existing literature used this technology. Even in the middle of their density range, most phographic emulsions have logarithmic density functions. And tube-based video cameras are not a lot better. Among the common detectors, photodiodes, photomultipliers, CMOS and CCD sensors all offer reasonably linear responses.
Never mind, as you say, the gamma functions of offset printing or different displays.
I would emphasize, though, that CCD sensors really do have almost uncannily linear responses when used properly. As Ken Spring points out, CCDs are more linear than most of the devices that can be used to measure their linearity. Truly a remarkable technology. And this behavior is one of the major reasons that scientists love CCD cameras. -
Yes, but CCDs are about the only nearly linear step in a whole series. It's great if you're doing quantitative imaging, but as soon as you display the thing, you're twisting the pixel values.
My work and my students' work right now is all visual, so that's what we're concerned with. -
For me -- an amateur -- there's not enough here to enable me to understand what this is about. But I'm curious and would welcome more explicit explanation.
I assume the point is not copyright or plagiarism, that is, I assume the potential problem being addressed is not just using someone else's work or image without attribution or permission -- and I fully understand that altering someone else's image and then claiming it's your own (explicitly or implicitly) would be a bad practice.
Beyond that, though, I'm not sure I understand what the issues are.
If the objective is to illustrate or depict something accurately, why isn't the question simply whether the finished image does that? If this is the question, why would it matter whether the ultimate image has been produced by modification of a preexisting image ... if it is accuate in the end? Similarly, if accurate reproduction/depiction is the objective, why would it matter if the modifications to an existing image are only partial -- again assuming that what is ultimately produced accurately reflects/depicts the object to be illustrated? This same question also relates to the prohibitions against using "paintbrush" and similar tools, or cutting and pasting. (I do, of course, see that disclosing how you prepared your image is a good idea, and also disclosing any modifications made to preexisting images, and explanation of what has been enhanced/changed and why and how.)
It's possible that I've missed the point here ... but that's the question I started with.#: Posted by on 04/22 at 10:23 AM -
Hang on a second. Image processing can be a useful tool for revealing details that are not immediately apparent. Imposing arbitrary rules around the use of these tools might wind up restricting the utility of these tools. I'd propose that the primary rule should be disclosure, listing all the steps that have been used to process the image.
#: Posted by on 04/22 at 10:27 AM
-
Those aren't arbitrary rules. They are rules designed to prevent people from putting in stuff that isn't there, or hiding data that doesn't fit an expectation. There is nothing wrong about a little edge enhancement, a bit of contrast enhancement, some tasteful pseudocoloring (all with disclosure, of course), but lassoing that ugly artifact in the top left corner of your purty picture and airbrushing it away? That's a no-no.
Right, Jeff, it's because things like gels and photomicrographs already have a bit of the unfamiliar and otherworldly to them, and it's way to easy for even an amateur to touch them up in misleading ways. It's also our primary data: fudging evidence is bad. -
OK. Arbitrary was perhaps a poor choice of word. My point is that many types of image processing may reveal details that are not immediately apparent, and that the best way to judge the quality of the results of image processing techniques is to reveal the actual techniques used. I'd say that the best rule to put around image processing is complete disclosure, whether it's contrast changes or edge detection.
#: Posted by on 04/22 at 11:08 AM
-
Well, yes -- those are OK. That's what the article is saying, though, is that operations applied to the whole image are acceptable, but tweaking bits and pieces are not.
They give examples: for instance, increasing contrast to make all the bands in a gel clearer is OK, but dodging away that annoying diffuse band that blurs the sharp edges of your protein X is right out. -
Ah, this takes me back to my graduate school days, before digital cameras, before our lab even used Adobe Photoshop (although we did have several computers in the lab). Back in those days, we took photos of gels, cut them out and matted them with labels, and then took another picture of the finished product to use as figures for a journal. It was a tedious, laborious process that often required repetition to get it right.
Believe it or not, that was only the early 1990's when we still did that. That was the way all the figures for my thesis (excluding the ones which were simple graphs, for which we used Kaleidagraph or Microsoft Excel, of course) were done in 1993. - You had photography? We had to knack our own chisels from flint, plane a board, and carve our own woodcut blocks. And we liked it.
-
There are more types of images used in medicin/biology than photomicrographs, eg CAT/MRI-images, electron photomicrograps etc, do these rules fit equally well to them? For example, 3D CAT:s typically color different tissues in different pseudocolors, is that OK, or must only simple b/w section-images be used?
If the paintbrush have to be explicitly prohibited, wouldn't it make sense too to prohibit any form of lossy image compression?
Lastly, for full disclosure, why not require that original images (in digital form, if applicable) be sent along with any paper submitted for review? After publication, these images should be made available to any peer wishing to review the data.#: Posted by on 04/22 at 12:06 PM -
A perfect application for a domain-specific language for scientific graphics manipulation. The best documentation of the graphics enhancement would be a textual snippet of code describing the manipulation. Scientists would know with absolute precision what transformations had been applied, and could even replicate the manipulation by downloading the original graphic and typing in the code.
To ease the job of the user, you could design a GUI editing environment that provided the "acceptable" subset of Photoshop-type tools, recorded the manipulations, and spit out the equivalent code.
Provided somebody else did the hard design thinking about which manipulations were acceptable, a bright CS undergrad who had taken a programming languages class and a graphics class could do this as a term project. Of course, if you're going to make this language a standard in the scientific community, you wouldn't want to leave it up to an undergrad, and you'd want to set up some infrastructure to keep the software maintained, etc.
(Actually, something quite close to the above exists, minus the GUI tool: look at the ImageMagick batch image conversion software.) -
"information" can be pulled out of anything. Generating toxic ooze textures for video games is fun to photoshop, when random noise can be blurred and edge sharpened and level adjusted, dropped into rgb channels and tiled.
What I find interesting is that modern astrophotography and image processing seem nearly indistinguishable; details are available only at certain wavelengths, and visualizing that detail requires transposing that data into wavelengths the eye can process. The raw data is there for all to process, and histogram settings are published along with the images so that anybody is free to tweaze it themselves, even if they do pull faces out of the noise of the Cydonian plain.
ISTR some minor controversy over processes employed by the photographer who produced the touring show and book, Full Moon, Michael Light, with his selective blurring and sharpening on seldom seen Apollo mission photographs. I suppose such esthetic tweaking can be justified under the category of science outreach. -
Personally, if I have an air bubble on my slide that is in the image field but not part of the specimen, I don't see the problem with removing it in Photoshop. If I can crop it out, fine. If not, I remove it. Yes, it is deceptive. But it is not deceptive with respect to the specimen. You are not asking the audience to draw scientific conclusions about the space around your specimen. Moving your hungry cursor over onto the specimen proper is a no-no.
#: Posted by on 04/22 at 12:15 PM
- p.s. I forgot to mention, there are also a number of declarative graphics description/manipulation languages out there, in academia and elsewhere (see, e.g., Conal Elliott's Pan system, not to mention PostScript and its cousins), but off the top of my head I'm not familiar with any that are designed exactly with this application in mind.
-
I don't quite understand why "All enhancements must be global." When I am taking photos thru my microscope I frequently use uneven illumination so that there will be shadows to bring out the contrast. But an unwanted result of this is that a part of a picture may turn out too dark than the rest. In a case like this, I don't see why it would be "unethical" to select the dark part & lighten it, while keeping the rest the same. I am not adding anything other than "light".
Likewise, why can't I selectively sharpen a blurry part? Why is that different than doing a montage? -
The rules are there to prevent misleading manipulation. In order for the rules to work, they have to apply to everyone, the honest as well as the dishonest. It seems quite clear to me that requiring global changes tends to lessen the liklihood of misleading manipulations by someone less ethical than those who have commented here. Many of the manipulations mentioned here that don't meet the requirements but that do not alter the meaning of the image are cosmetic. There is little reason to allow cosmetic changes that violate the rules because it opens the door to changes in the substance of an image. If I were a journal editor and I learned that someone had not followed the guidelines, however innocently, I would require that a new, unaltered image be submitted. If the image requires substantial manipulation in order to show an effect, I would also have to ask whether the effect is actually present in the image. It is hard to manipulate an image in any way without altering the information contained in the image.
#: Posted by on 04/22 at 01:57 PM
-
Dodging & burning-in were standard techniques in dark room photography. In fact, a classic book on scientific photography (Alfred Blaker, Handbook for Scientific Photography, 1977) demonstrates how to do what the author called "differential printing".
If it was ethical to manipulate parts of a photograph in the dark room, it is ethical to manipulate parts of a digital image.
As far as I am concerned, the case is closed. -
I wrote a policy on image manipulation for the <A HREF ="http://www.wehi.edu.au">Walter and Eliza Hall Institute</a> where I worked as the graphics head before I became a poorly paid academic (but I repeat myself).
In that policy, I suggested that if any part of the image s modified, it must be explicitly mentioned in the accompanying text. Any excisions, such as intervening lanes in a Western, had to be marked as a white space, the way a quote would show ellipses to mark excised text. And any unwanted marks or artifacts had to be kept - it was dishonest to imply you were better at the technique than you really were, and who knew what might become important information in the future?
This worked well to prevent people from going mad with PhotoShop to rejigger their data. We also had a policy that all original images as captured must be kept on a backup site for seven years, the way other data. Modified images were not kept in that system, and if the data was captured using one of the main instruments of the institute, it was automatically archived.#: Posted by John Wilkins on 04/22 at 06:13 PM -
I assume the point is not copyright or plagiarism, that is, I assume the potential problem being addressed is not just using someone else's work or image without attribution or permission -- and I fully understand that altering someone else's image and then claiming it's your own (explicitly or implicitly) would be a bad practice.
Beyond that, though, I'm not sure I understand what the issues are.
The point is to not misrepresent the data. The old HIV tests used bands on a strip of paper. If the band(s) was/were there, you had HIV. If not, then you didn't have HIV infection. What if the band is maybe sort of there? Can I increase the contrast to make something appear that isn't really there. Or crop away the parts that I don't like.
ersonally, if I have an air bubble on my slide that is in the image field but not part of the specimen, I don't see the problem with removing it in Photoshop.
Wha?! You're advisor didn't make you reshoot speciments until you got an aesthetically pleasing image?!
If I want to show targeting of a green fluorescent protein tag to a specific cell compartment, I may have to look at 1000-100000 cells to get the clearest picture. That problem arises if I only find 1 cell in 10,000 that shows what I want while the other 9,999 show nothing like it. -
"If it was ethical to manipulate parts of a photograph in the dark room, it is ethical to manipulate parts of a digital image."
This approaches begging the question.
It is certainly ethical to manipulate a photograph if you are making an aesthetic statement. However, scientists are expected not to manipulate data except under strictly controlled and described conditions. An image in a scientific paper that is presented as evidence must not be manipulated except under those same conditions. The journal is simply stating what the conditions are. The policy mentioned is an attempt to prevent intentional or inadvertent misrepresentation. What's wrong with such a policy?#: Posted by on 04/22 at 10:57 PM -
A number of really interesting issues here.
First: linearity and display. I agree with you on this one, PZ. Any image that is displayed on a noncalibrated monitor is going to have the wrong gamma, the wrong, white point, etc. Photographic film emulsions have the characteristic densitometric responses that they do in large part because these responses are tailored to yield images that reproduce clearly. Scientific CCDs produce raw data that is not well suited to printing or display, though it is well-suited to quantitative analysis. A pet peeve in many current papers is the unnecessary presentation of extreme, nearly or actually thresholded data, rather than the presentation of the dynamic range present in the original data. There oughtta be a law...
Second: background correction. There are a lot of ways to do this that are in fact global. In some cases, the interesting science is simply impossible without background subtraction or other shading corrections. These applications include quantitative analysis of intensity, ratiometric imaging of ion-selective dyes, and nanometer-scale tracking of particles using video-enhanced differential interference contrast.
At lower magnificaiton, for a Brilliant Blue-stained protein gel, one can scan in color (RGB) and all of the data will be in the red and green channels, since the blue stain absorbs red and green light. The blue channel can be subtracted from the others -- globally -- to remove nonspecific background such as dust or bubbles. Similar techniques using an infrared channel are used for dust/scratch rejection on mosth high-end film scanners. This is totally valid, and it is a *global* change.
But it is *not* cool to locally remove a bubble, in my opinion; here I do not agree with what (I think) others here have argued. If the bubble is so important that it needs to be removed, then it is worth doing the experiment again, to obtain a micrograph that is of "publication" technical quality.
Local cloning, dodging, burning, etc. are unacceptable for presentation of primary data except in specific circumstances. Under such circumstances it is the investigator's responsibility to specify which manipulations were performed and to justify the requirement for these manipulations. It is also worth noting that many, many scientists have *no* idea of what their instruments are doing to their data. On one occasion I have showed a colleague how to apply an unsharp mask (this is, oddly enough, a sharpening filter) to a scanned image of a gel. He was appalled that I'd modify an image in this way until I asked him to show me how he scanned his gels. Of course, he had not noticed that his scanning software was applying an unsharp mask by default! We won't even venture into the treacherous marshlands of "quantitative" western blots (mm'kay?).
Third: there is a persistent notion that the ability to cheat by image manipulation is a new development. It certainly is not. Most of the techniques that can lead to questionable practices have existed since the dawn of silver emulsion photography. It's true that they are easier now, but the required skill was not difficult to obtain in the silver halide era. Indeed, anyone who spent time doing serious professional or amateur darkroom work had access to these techniques. And there were a LOT of people who did have these skills.
Fourth: it is probably not practical to build a standardized operational grammar or language for image manipulation simply because of the huge diversity of image enhancement, sharpening, rendering, and deconvolution algorithms and implementations presently employed in biological imaging.
There's a lot more to say, but those are the points that really stand out for me in this conversation. -
Wilkins's point about removing blemishes is *very* sound. It is not honest to make it look as though you have bettter hands than you actually do! A case in point can be found in the spectacular new paper on genetic reversion at the Arabidopsis HOTHEAD locus*. The paper includes a DNA blot hybridization of the type popularized by E.M. Southern. This figure is critical to interpretation of the data. The data shown are convincingly good, but not perfect. There are the migrating DNA bands in locations consistent with the authors' interpretation, but there are also a number of spots of nonspecific hybridization. Crud. Dirt.
Now, ten or fifteen years ago Southern blots were much more commonly employed than they are today, and a typical paper in Cell or Nature would have immaculate blots. People were doing a lot of blots and they were really good at it. Today, the use of polymerase chain reaction (PCR) has made Southern blots obsolete in many cases. People are out of practice. And so a potentially paradigm-cracking paper has a blot that is not immaculate. It is good enough. And it is clear that no one thought they had to Photoshop, or dodge, or retouch out those specs of crud.
* Lolle et al. 2005, Genome-wide non-mendelian inheritance of extra-genomic information in Arabidopsis. Nature 434:505. -
A colleague was involved in research recently where he discovered some extra bands on his Westerns that had never been reported before in the literature, and all the published images showed immaculate bands, which we just couldn't reproduce - turns out that these extra bands are highly relevant to the disease process we were studying - now I don'tknow if everyone else was chopping those bands out, but it shows the risks.
#: Posted by on 04/23 at 04:57 AM
-
Alex has it exactly right. There are ways to improve a digital image that are objective and can be uniformly applied to the entire image, and those are acceptable; I do shading correction with background subtraction all the time. But tweaking out bits of dirt or bubbles because they are aesthetically objectionable is not something you should do.
Another way to think of it is that an image is a 2-D table of numbers. If you were presenting a table of measurements in a paper, do you think it would be OK to take the number "3" from one cell and replace it with "71", because it would look better? -
A potentially pernicious sort of image manipulation is showing just a small portion of a gel or blot, containing only the "important" bands.
I recently dealt as journal editor with a manuscript in which a picture of a complete blot contained additional strong bands beyond those expected, indicating that something was very wrong with the experiment. A reviewer pointed this out. The response of the authors was to resubmit a revised version in which the region with the "wrong" bands was cut off and only the "right" bands were shown! This led to an angry rejection letter, but if the authors had initially submitted the cropped version no one would have realized there was a problem.
Concerning global adjustments: A colleague says one can take an immunofluorescence micrograph produced with any <i>preimmune<i> serum, and by sufficient adjustment of contrast and brightness produce what looks like a perfectly believable "specific" staining. He is very likely right.
At some level one has to trust that authors are both competent and honest.#: Posted by on 04/23 at 12:01 PM -
"At some level one has to trust that authors are both competent and honest."
And that would be your first mistake.#: Posted by on 04/23 at 12:48 PM -
This is somewhat off topic, but here is an interesting article on color in astronomical images.
#: Posted by on 04/23 at 02:43 PM
-
But it is *not* cool to locally remove a bubble, in my opinion; here I do not agree with what (I think) others here have argued. If the bubble is so important that it needs to be removed, then it is worth doing the experiment again, to obtain a micrograph that is of "publication" technical quality.
If it is possible to do the experiment quickly, I agree. If, however, your collaborator would have to breed some conditional knockout mice again, then I would remove the bubble, on condition that the bubble does not intrude upon the specimen itself.
Alex, John and Dr. Meyerz: What do you think about modifying one panel in a figure, but not another? For example, if you have bright field images of two different specimens with slightly different background colours, would it be acceptable to alter one background colour to match the other?#: Posted by on 04/23 at 04:42 PM -
In my lab I most likely would not permit the removal of the bubble. If it really had to go, I would state in the figure caption -- not in the figure legend -- that a bubble had been retouched. This is another way to look at the problem: would you be uncomfortable with telling readers, in the caption, how the image was produced? Take a look at the Southern blot in the HOTHEAD paper. It's not pretty, but the important thing is the result, not the absence of technical flaws.
All this said, the developmental biologists here know that much of the old literature has no photos at all, but gorgeous, lucid, idealized pen-and-ink drawings. These generally reflect both an effort to portray the biological observations as lucidly as possible and, often, a serious aesthetic sensibility as well. And you can bet that many camera lucida drawings -- primary data -- reasonably omitted bubbles in the preps. I don't claim that these judgments are trivial to make, but I can say what I'd sign off on in my own lab.
With respect to color-correcting individual brightfield panels that are part of a set, I'd generally have no problem at all with this, provided that doing so would not influence any quantitative treatments required to interpret the data. Obtaining color constancy is a really challenging, as anyone who's ever been involved in commercial photography knows.
By the way there is a fascinating thread over at Edward Tufte's wonderful site, on the subject of obfuscation in technical argument. Another thread deals with documented and undocumented image manipulations. These, and many of the other threads there, will be of interest to many Pharyngula readers. - Whoops, I meant state it in the caption, not bury it in the Materials and Methods. Though that's still better than not saying it at all.
-
It isn't very hard to see why local modifications are a no-no: when you take a photo, what you are actually doing is acquiring data. To help understanding, the data can be subject to mathematical operations before being displayed. That's what global transforms do. Perfectly acceptable, as long as it is documented. But local image manipulation is another beast altogether: you are destroying data to make your experimental results look better. Saying 'that air bubble isn't meaningful, therefore I can wipe it off' is akin to saying 'those readings are clearly anomalous, and therefore I don't need to present them'.
#: Posted by on 04/24 at 11:07 AM
-
This discussion is causing me to rethink an approach I am taking in one of my projects, and I would like to ask for your advice in how I think these things through.
I am collaborating on a team which is publishing a cytology atlas for certain cells for mammalian species whose cytology has not been documented to date. The intended audience does not necessarily have specialized training in biology--for example, wildlife conservation workers in the developing world with sporadic or no access to veterinary care--so I am assuming nothing when it comes to prior knowledge of cell biology. In order to highlight the differences between types of cells for pedagogical purposes, I've been using Photoshop tools to increase the contrast and edges, remove extraneous objects that are not under discussion (leukocytes, debris), and to juxtapose highly representative examples of different types of cells for contrast with each other.
I thought that the teaching value of these interventions justified the image manipulation, which is why I was proceeding in this way. But the discussion is raising issues that I had not considered: for example, the point about not pretending to be a better photographer than you really are. If I were a better photomicrographer, perhaps I would not have to increase contrast and edges--they would already be that way in the photograph. So maybe it is not appropriate to do so in Photoshop.
Perhaps the other issues aren't applicable, because I am not presenting the cells as research results, but as teaching examples. Or perhaps they are applicable, because they are what others will see in their research or clinical results, and so I should not move things around or remove extraneous material to emphasize contrast, because that gives a false ideal of what they will see when they undertake the work. Or maybe it is still ok, as long as I explicitly say what I did for each image.
So now I am rethinking my approach to the atlas, and would be interested in hearing your opinions as to whether the inteventions I described would constitute appropriate image manipulation.#: Posted by on 04/24 at 11:45 AM -
"Saying 'that air bubble isn't meaningful, therefore I can wipe it off' is akin to saying 'those readings are clearly anomalous, and therefore I don't need to present them'."
Absolute rubbish. Obviously it depends on what you are doing, but if I were to cut out or cover up an air bubble in one of my sections (which I probably wouldn't need to do), it would be to make a pretty picture for a paper. The actual analysis I do would have nothing to do with that air bubble - who cares that my mounting technique is sloppy? Maybe the image shows all your cell types nicely in one frame, that is an aesthetic and practical judgement, it has absolutely no bearing on the actual analysis.#: Posted by on 04/24 at 12:56 PM -
"So now I am rethinking my approach to the atlas, and would be interested in hearing your opinions as to whether the inteventions I described would constitute appropriate image manipulation."
I'm not sure I can judge, because it is certainly the case that cruddy examples can really confuse people (I have a really crap human atlas I'm working from at the moment, very annoying), but on the other hand, overly perfect examples can make you think that you are doing things wrong. I would certainly mention any major manipulations (very good contrast in examples might make people wonder about their morphology say).#: Posted by on 04/24 at 01:03 PM -
RS,
Yes, so long as you realise that a doctored photo cannot be used as a basis for any scientific claim; that you could just as well replace it by a page 3 girl without it having any consequences on your analysis. Not that I would suggest doing so,of course.#: Posted by on 04/24 at 01:50 PM -
SirL,
depends what that scientific claim is - if the claim is, look, these are the cell types you find in this area, then why not? Then you go on to report that you counted them and what you found. So it isn't part of your primary data, but it is conveying a piece of scientific information.
Personally, the reason I wouldn't use a doctored photo is because it (if it came out) might cast doubt on your results proper. So its more about the image than the reality.#: Posted by on 04/24 at 02:57 PM -
Alex writes: Fourth: it is probably not practical to build a standardized operational grammar or language for image manipulation simply because of the huge diversity of image enhancement, sharpening, rendering, and deconvolution algorithms and implementations presently employed in biological imaging.
I find that pretty surprising, and probably overly pessimistic. Certain kinds of scientific imaging might be hard to support, but I suspect you could design a language that served the vast majority of researchers' needs in a relatively straightforward way. Most of the image manipulations offered by Photoshop, for example, can be described compactly by feeding a characteristic matrix into one of a handful of standard algorithms. The ones that cannot be so characterized tend to be things that you probably shouldn't be doing with a scientific image anyway.
I like PZ's comparison of an image to a table of numbers. Image manipulation really is the application of some algorithm to a table of numbers. Scientists do this all the time --- it's called statistical analysis --- but they use standard algorithms with a few well-known parameters. I have difficulty believing that the graphics algorithms that biologists require cannot be standardized and parameterized in the same way.
Also, you could always punt and make the language Turing-complete, which would permit the definition of arbitrary algorithms over the input. Standard libraries of such routines could be defined and referenced, so that researchers wouldn't have to include the full code in every paper. -
'Most of the image manipulations offered by Photoshop, for example, can be described compactly by feeding a characteristic matrix into one of a handful of standard algorithms'.
Much simpler than that, in fact. Many operators can be expressed in a mask which convolved with the image matrix. Gaussian blurring, for instance, is precisely that: the convolution of a Gaussian with the image. Quite a lot of useful stuff, such as edge detection, can be done this way.
But then there a lot of other useful tools too. Hough transforms spring to mind.
'The ones that cannot be so characterized tend to be things that you probably shouldn't be doing with a scientific image anyway'.
There's some real state-of-the-art image processing going on in medical imaging. I don't think most of it would easily reduce itself to a neat formula.
'I like PZ's comparison of an image to a table of numbers. Image manipulation really is the application of some algorithm to a table of numbers'
We like to think of images as signals. Quite a bit of image processing turns out to be two-dimensional digital signal processing.#: Posted by on 04/24 at 03:59 PM -
I think that using transforms, touching up images and so on for the purpose of a teaching text like a cell atlas is perfectly OK. It is not research, and the material here is for propaedeutic purposes only.
In research, if you do modiy an image, even globally, you must note this in the caption, and preferably also in the Materials and Methods, and keep an unmodified copy for sending to people for reanalysis if required.
My two cents. I've never done a day's research in my life.#: Posted by John Wilkins on 04/24 at 05:29 PM -
Cog, my concern has at as much - more, in fact - to do with the diversity of software implementations as the diversity of algorithms.
Too many of the vendors that support microscopy image acquisition processing have shown themselves unwilling to support even simple things, like the Open Microscopy Environment's file and metadata standards. In this heavily proprietary environment, standardizing image manipulation procedures will not happen unless the user base demands it. And from what I can tell, the community is very, very far from speaking with the unified voice that would be required to effect such a program. Most biologists (PI's, grad students, postdocs) are interested in just getting this week's data, the current paper submitted, etc. By way of illustration, very many, possibly a majority, are people who think that PowerPoint is a suitable software environment for plotting, figure layout, and even image processing.
So I'm not saying that it will not happen. Just that it will not happen in the foreseeable future. -
Raven, for teaching purposes, it seems to me that the images should show the characteristics in question. That might justify removing unrelated parts of the image. However, if by removing the unrelated parts it makes it harder to identify the characteristics in question (because, for example, the thing you are trying to show might never be present in nature with the appearance it has after the manipulation) you might be reducing the teaching value of the images. Maybe showing images before and after would help. That would allow you to show the characteristics as they might be seen in an actual sample, and also to show the specific features that distinguish them. I do think that images used for teaching are different from images used in a research paper.
#: Posted by on 04/24 at 11:11 PM