#22712: Alex Merz — 04/22  at  09:41 AM
Good point about linearity. It is not as though photographic film has a linear density response to illumination, and most of the existing literature used this technology. Even in the middle of their density range, most phographic emulsions have logarithmic density functions. And tube-based video cameras are not a lot better. Among the common detectors, photodiodes, photomultipliers, CMOS and CCD sensors all offer reasonably linear responses.

Never mind, as you say, the gamma functions of offset printing or different displays.

I would emphasize, though, that CCD sensors really do have almost uncannily linear responses when used properly. As Ken Spring points out, CCDs are more linear than most of the devices that can be used to measure their linearity. Truly a remarkable technology. And this behavior is one of the major reasons that scientists love CCD cameras.

#22714: PZ Myers — 04/22  at  09:51 AM
Yes, but CCDs are about the only nearly linear step in a whole series. It's great if you're doing quantitative imaging, but as soon as you display the thing, you're twisting the pixel values.

My work and my students' work right now is all visual, so that's what we're concerned with.

#22720: — 04/22  at  10:23 AM
For me -- an amateur -- there's not enough here to enable me to understand what this is about. But I'm curious and would welcome more explicit explanation.

I assume the point is not copyright or plagiarism, that is, I assume the potential problem being addressed is not just using someone else's work or image without attribution or permission -- and I fully understand that altering someone else's image and then claiming it's your own (explicitly or implicitly) would be a bad practice.

Beyond that, though, I'm not sure I understand what the issues are.

If the objective is to illustrate or depict something accurately, why isn't the question simply whether the finished image does that? If this is the question, why would it matter whether the ultimate image has been produced by modification of a preexisting image ... if it is accuate in the end? Similarly, if accurate reproduction/depiction is the objective, why would it matter if the modifications to an existing image are only partial -- again assuming that what is ultimately produced accurately reflects/depicts the object to be illustrated? This same question also relates to the prohibitions against using "paintbrush" and similar tools, or cutting and pasting. (I do, of course, see that disclosing how you prepared your image is a good idea, and also disclosing any modifications made to preexisting images, and explanation of what has been enhanced/changed and why and how.)

It's possible that I've missed the point here ... but that's the question I started with.

#22721: — 04/22  at  10:27 AM
Hang on a second. Image processing can be a useful tool for revealing details that are not immediately apparent. Imposing arbitrary rules around the use of these tools might wind up restricting the utility of these tools. I'd propose that the primary rule should be disclosure, listing all the steps that have been used to process the image.

#22723: PZ Myers — 04/22  at  10:43 AM
Those aren't arbitrary rules. They are rules designed to prevent people from putting in stuff that isn't there, or hiding data that doesn't fit an expectation. There is nothing wrong about a little edge enhancement, a bit of contrast enhancement, some tasteful pseudocoloring (all with disclosure, of course), but lassoing that ugly artifact in the top left corner of your purty picture and airbrushing it away? That's a no-no.

Right, Jeff, it's because things like gels and photomicrographs already have a bit of the unfamiliar and otherworldly to them, and it's way to easy for even an amateur to touch them up in misleading ways. It's also our primary data: fudging evidence is bad.

#22725: — 04/22  at  11:08 AM
OK. Arbitrary was perhaps a poor choice of word. My point is that many types of image processing may reveal details that are not immediately apparent, and that the best way to judge the quality of the results of image processing techniques is to reveal the actual techniques used. I'd say that the best rule to put around image processing is complete disclosure, whether it's contrast changes or edge detection.

#22727: PZ Myers — 04/22  at  11:41 AM
Well, yes -- those are OK. That's what the article is saying, though, is that operations applied to the whole image are acceptable, but tweaking bits and pieces are not.

They give examples: for instance, increasing contrast to make all the bands in a gel clearer is OK, but dodging away that annoying diffuse band that blurs the sharp edges of your protein X is right out.

#22729: Orac — 04/22  at  11:54 AM
Ah, this takes me back to my graduate school days, before digital cameras, before our lab even used Adobe Photoshop (although we did have several computers in the lab). Back in those days, we took photos of gels, cut them out and matted them with labels, and then took another picture of the finished product to use as figures for a journal. It was a tedious, laborious process that often required repetition to get it right.

Believe it or not, that was only the early 1990's when we still did that. That was the way all the figures for my thesis (excluding the ones which were simple graphs, for which we used Kaleidagraph or Microsoft Excel, of course) were done in 1993.

#22730: PZ Myers — 04/22  at  12:02 PM
You had photography? We had to knack our own chisels from flint, plane a board, and carve our own woodcut blocks. And we liked it.

#22731: — 04/22  at  12:06 PM
There are more types of images used in medicin/biology than photomicrographs, eg CAT/MRI-images, electron photomicrograps etc, do these rules fit equally well to them? For example, 3D CAT:s typically color different tissues in different pseudocolors, is that OK, or must only simple b/w section-images be used?

If the paintbrush have to be explicitly prohibited, wouldn't it make sense too to prohibit any form of lossy image compression?

Lastly, for full disclosure, why not require that original images (in digital form, if applicable) be sent along with any paper submitted for review? After publication, these images should be made available to any peer wishing to review the data.

#22732: Cog — 04/22  at  12:13 PM
A perfect application for a domain-specific language for scientific graphics manipulation. The best documentation of the graphics enhancement would be a textual snippet of code describing the manipulation. Scientists would know with absolute precision what transformations had been applied, and could even replicate the manipulation by downloading the original graphic and typing in the code.

To ease the job of the user, you could design a GUI editing environment that provided the "acceptable" subset of Photoshop-type tools, recorded the manipulations, and spit out the equivalent code.

Provided somebody else did the hard design thinking about which manipulations were acceptable, a bright CS undergrad who had taken a programming languages class and a graphics class could do this as a term project. Of course, if you're going to make this language a standard in the scientific community, you wouldn't want to leave it up to an undergrad, and you'd want to set up some infrastructure to keep the software maintained, etc.

(Actually, something quite close to the above exists, minus the GUI tool: look at the ImageMagick batch image conversion software.)

#22733: Ken Cope — 04/22  at  12:13 PM
"information" can be pulled out of anything. Generating toxic ooze textures for video games is fun to photoshop, when random noise can be blurred and edge sharpened and level adjusted, dropped into rgb channels and tiled.

What I find interesting is that modern astrophotography and image processing seem nearly indistinguishable; details are available only at certain wavelengths, and visualizing that detail requires transposing that data into wavelengths the eye can process. The raw data is there for all to process, and histogram settings are published along with the images so that anybody is free to tweaze it themselves, even if they do pull faces out of the noise of the Cydonian plain.

ISTR some minor controversy over processes employed by the photographer who produced the touring show and book, Full Moon, Michael Light, with his selective blurring and sharpening on seldom seen Apollo mission photographs. I suppose such esthetic tweaking can be justified under the category of science outreach.

#22734: — 04/22  at  12:15 PM
Personally, if I have an air bubble on my slide that is in the image field but not part of the specimen, I don't see the problem with removing it in Photoshop. If I can crop it out, fine. If not, I remove it. Yes, it is deceptive. But it is not deceptive with respect to the specimen. You are not asking the audience to draw scientific conclusions about the space around your specimen. Moving your hungry cursor over onto the specimen proper is a no-no.

#22735: Cog — 04/22  at  12:17 PM
p.s. I forgot to mention, there are also a number of declarative graphics description/manipulation languages out there, in academia and elsewhere (see, e.g., Conal Elliott's Pan system, not to mention PostScript and its cousins), but off the top of my head I'm not familiar with any that are designed exactly with this application in mind.

#22741: Aydin — 04/22  at  01:31 PM
I don't quite understand why "All enhancements must be global." When I am taking photos thru my microscope I frequently use uneven illumination so that there will be shadows to bring out the contrast. But an unwanted result of this is that a part of a picture may turn out too dark than the rest. In a case like this, I don't see why it would be "unethical" to select the dark part & lighten it, while keeping the rest the same. I am not adding anything other than "light".

Likewise, why can't I selectively sharpen a blurry part? Why is that different than doing a montage?

#22743: — 04/22  at  01:57 PM
The rules are there to prevent misleading manipulation. In order for the rules to work, they have to apply to everyone, the honest as well as the dishonest. It seems quite clear to me that requiring global changes tends to lessen the liklihood of misleading manipulations by someone less ethical than those who have commented here. Many of the manipulations mentioned here that don't meet the requirements but that do not alter the meaning of the image are cosmetic. There is little reason to allow cosmetic changes that violate the rules because it opens the door to changes in the substance of an image. If I were a journal editor and I learned that someone had not followed the guidelines, however innocently, I would require that a new, unaltered image be submitted. If the image requires substantial manipulation in order to show an effect, I would also have to ask whether the effect is actually present in the image. It is hard to manipulate an image in any way without altering the information contained in the image.

#22748: Aydin — 04/22  at  03:23 PM
Dodging & burning-in were standard techniques in dark room photography. In fact, a classic book on scientific photography (Alfred Blaker, Handbook for Scientific Photography, 1977) demonstrates how to do what the author called "differential printing".

If it was ethical to manipulate parts of a photograph in the dark room, it is ethical to manipulate parts of a digital image.

As far as I am concerned, the case is closed.

#22762: John Wilkins — 04/22  at  06:13 PM
I wrote a policy on image manipulation for the <A HREF ="http://www.wehi.edu.au">Walter and Eliza Hall Institute</a> where I worked as the graphics head before I became a poorly paid academic (but I repeat myself).

In that policy, I suggested that if any part of the image s modified, it must be explicitly mentioned in the accompanying text. Any excisions, such as intervening lanes in a Western, had to be marked as a white space, the way a quote would show ellipses to mark excised text. And any unwanted marks or artifacts had to be kept - it was dishonest to imply you were better at the technique than you really were, and who knew what might become important information in the future?

This worked well to prevent people from going mad with PhotoShop to rejigger their data. We also had a policy that all original images as captured must be kept on a backup site for seven years, the way other data. Modified images were not kept in that system, and if the data was captured using one of the main instruments of the institute, it was automatically archived.

#22766: madbard — 04/22  at  07:46 PM
I assume the point is not copyright or plagiarism, that is, I assume the potential problem being addressed is not just using someone else's work or image without attribution or permission -- and I fully understand that altering someone else's image and then claiming it's your own (explicitly or implicitly) would be a bad practice.

Beyond that, though, I'm not sure I understand what the issues are.

The point is to not misrepresent the data. The old HIV tests used bands on a strip of paper. If the band(s) was/were there, you had HIV. If not, then you didn't have HIV infection. What if the band is maybe sort of there? Can I increase the contrast to make something appear that isn't really there. Or crop away the parts that I don't like.

ersonally, if I have an air bubble on my slide that is in the image field but not part of the specimen, I don't see the problem with removing it in Photoshop.

Wha?! You're advisor didn't make you reshoot speciments until you got an aesthetically pleasing image?!

If I want to show targeting of a green fluorescent protein tag to a specific cell compartment, I may have to look at 1000-100000 cells to get the clearest picture. That problem arises if I only find 1 cell in 10,000 that shows what I want while the other 9,999 show nothing like it.

#22774: — 04/22  at  10:57 PM
"If it was ethical to manipulate parts of a photograph in the dark room, it is ethical to manipulate parts of a digital image."

This approaches begging the question.

It is certainly ethical to manipulate a photograph if you are making an aesthetic statement. However, scientists are expected not to manipulate data except under strictly controlled and described conditions. An image in a scientific paper that is presented as evidence must not be manipulated except under those same conditions. The journal is simply stating what the conditions are. The policy mentioned is an attempt to prevent intentional or inadvertent misrepresentation. What's wrong with such a policy?

#22782: Alex Merz — 04/23  at  12:21 AM
A number of really interesting issues here.

First: linearity and display. I agree with you on this one, PZ. Any image that is displayed on a noncalibrated monitor is going to have the wrong gamma, the wrong, white point, etc. Photographic film emulsions have the characteristic densitometric responses that they do in large part because these responses are tailored to yield images that reproduce clearly. Scientific CCDs produce raw data that is not well suited to printing or display, though it is well-suited to quantitative analysis. A pet peeve in many current papers is the unnecessary presentation of extreme, nearly or actually thresholded data, rather than the presentation of the dynamic range present in the original data. There oughtta be a law...

Second: background correction. There are a lot of ways to do this that are in fact global. In some cases, the interesting science is simply impossible without background subtraction or other shading corrections. These applications include quantitative analysis of intensity, ratiometric imaging of ion-selective dyes, and nanometer-scale tracking of particles using video-enhanced differential interference contrast.

At lower magnificaiton, for a Brilliant Blue-stained protein gel, one can scan in color (RGB) and all of the data will be in the red and green channels, since the blue stain absorbs red and green light. The blue channel can be subtracted from the others -- globally -- to remove nonspecific background such as dust or bubbles. Similar techniques using an infrared channel are used for dust/scratch rejection on mosth high-end film scanners. This is totally valid, and it is a *global* change.

But it is *not* cool to locally remove a bubble, in my opinion; here I do not agree with what (I think) others here have argued. If the bubble is so important that it needs to be removed, then it is worth doing the experiment again, to obtain a micrograph that is of "publication" technical quality.

Local cloning, dodging, burning, etc. are unacceptable for presentation of primary data except in specific circumstances. Under such circumstances it is the investigator's responsibility to specify which manipulations were performed and to justify the requirement for these manipulations. It is also worth noting that many, many scientists have *no* idea of what their instruments are doing to their data. On one occasion I have showed a colleague how to apply an unsharp mask (this is, oddly enough, a sharpening filter) to a scanned image of a gel. He was appalled that I'd modify an image in this way until I asked him to show me how he scanned his gels. Of course, he had not noticed that his scanning software was applying an unsharp mask by default! We won't even venture into the treacherous marshlands of "quantitative" western blots (mm'kay?).

Third: there is a persistent notion that the ability to cheat by image manipulation is a new development. It certainly is not. Most of the techniques that can lead to questionable practices have existed since the dawn of silver emulsion photography. It's true that they are easier now, but the required skill was not difficult to obtain in the silver halide era. Indeed, anyone who spent time doing serious professional or amateur darkroom work had access to these techniques. And there were a LOT of people who did have these skills.

Fourth: it is probably not practical to build a standardized operational grammar or language for image manipulation simply because of the huge diversity of image enhancement, sharpening, rendering, and deconvolution algorithms and implementations presently employed in biological imaging.

There's a lot more to say, but those are the points that really stand out for me in this conversation.

#22783: Alex Merz — 04/23  at  12:40 AM
Wilkins's point about removing blemishes is *very* sound. It is not honest to make it look as though you have bettter hands than you actually do! A case in point can be found in the spectacular new paper on genetic reversion at the Arabidopsis HOTHEAD locus*. The paper includes a DNA blot hybridization of the type popularized by E.M. Southern. This figure is critical to interpretation of the data. The data shown are convincingly good, but not perfect. There are the migrating DNA bands in locations consistent with the authors' interpretation, but there are also a number of spots of nonspecific hybridization. Crud. Dirt.

Now, ten or fifteen years ago Southern blots were much more commonly employed than they are today, and a typical paper in Cell or Nature would have immaculate blots. People were doing a lot of blots and they were really good at it. Today, the use of polymerase chain reaction (PCR) has made Southern blots obsolete in many cases. People are out of practice. And so a potentially paradigm-cracking paper has a blot that is not immaculate. It is good enough. And it is clear that no one thought they had to Photoshop, or dodge, or retouch out those specs of crud.

* Lolle et al. 2005, Genome-wide non-mendelian inheritance of extra-genomic information in Arabidopsis. Nature 434:505.

#22788: — 04/23  at  04:57 AM
A colleague was involved in research recently where he discovered some extra bands on his Westerns that had never been reported before in the literature, and all the published images showed immaculate bands, which we just couldn't reproduce - turns out that these extra bands are highly relevant to the disease process we were studying - now I don'tknow if everyone else was chopping those bands out, but it shows the risks.

#22790: PZ Myers — 04/23  at  06:30 AM
Alex has it exactly right. There are ways to improve a digital image that are objective and can be uniformly applied to the entire image, and those are acceptable; I do shading correction with background subtraction all the time. But tweaking out bits of dirt or bubbles because they are aesthetically objectionable is not something you should do.

Another way to think of it is that an image is a 2-D table of numbers. If you were presenting a table of measurements in a paper, do you think it would be OK to take the number "3" from one cell and replace it with "71", because it would look better?

#22805: — 04/23  at  12:01 PM
A potentially pernicious sort of image manipulation is showing just a small portion of a gel or blot, containing only the "important" bands.
I recently dealt as journal editor with a manuscript in which a picture of a complete blot contained additional strong bands beyond those expected, indicating that something was very wrong with the experiment. A reviewer pointed this out. The response of the authors was to resubmit a revised version in which the region with the "wrong" bands was cut off and only the "right" bands were shown! This led to an angry rejection letter, but if the authors had initially submitted the cropped version no one would have realized there was a problem.
Concerning global adjustments: A colleague says one can take an immunofluorescence micrograph produced with any <i>preimmune<i> serum, and by sufficient adjustment of contrast and brightness produce what looks like a perfectly believable "specific" staining. He is very likely right.
At some level one has to trust that authors are both competent and honest.