Denoise AI and the effect on detail

[myotis]

I've noticed AI denoise programs do odd things to textures in bird feathers, but this is a striking example of how AI denoising has affected the pattern of the
ommatidia (the small eyes that make up the big eyes) on this Hairy Dragonfly's compound eye.

The straight demosiaced raw file (NEF) looks very similar from either C1 and Lightroom (Lightroom not illustrated).

The Denoise AI file (Adobe Enhanced) generated in Lightroom, and the NEF file round tripped to Topaz PhotoAI from C1 both look similar to the straight demosaiced files from LR and C1.

BUT the DXO AI denoised files have a substantial change in the pattern of the ommatidia, making them appear to run in distinct lines. I tried both prime and prime XD, as well as altering the sharpening and other settings, but I couldn't get rid of the distinct line patterns. Which, misrepresent the actual patterns.

If you look at the other images, you can see a hint of of the ommatidia forming lines, but these aren't at the same angle as the lines created by DXO.

I'm not exactly sure what conclusions I draw from this. AI denoising gives some wonderful looking results with high ISOs, but equally, even though this is at 200%, for shots like this that might be used for some sort of scientific application, where the patterns shown might be important, it seems worth being aware of.

Superficially, I still feel DXO gives the best looking results, but I am beginning to think that Adobe Denoise AI might give more reliably "realistic" results when looking at things like the eye shown here and the details in feathers. C1 plus Topaz PhotoAI is also a possibility as Topaz seems to be much better than the last time I tried it at avoiding creating weird artifacts.

Note I have used the latest versions of C1, Adobe Denoise and Topaz Photo AI, but it was DXO PL 6 and not the latest version of DXOs denoising tools. This is the same version of denoising as is current in the latest version of DXO (version 7). DXO Pure Raw 4 has jumped a version ahead in terms of denoising, but won't be available in DXO PL until release 8 in the Autumn.

The problem with any of these AI denoising programs is that they aren't consistent, with the "best" choice of software varying between the different files tested. But this was an "impossible not to notice" difference, in results even at low magnification.



Edit: 26th April 2024

To add this, as explained, the Topaz example above was from the processed C1 Nef (ie on a TIFF). This was to see if a Topaz plus C1/LR approach could avoid the artefacting produces by DXO. DXO only works on Raw files.
So using Adobe deNoiseAI, or non AI denoising with LR or C1 with the AI denoising being passed to Topaz PhotoAI, both avoided the artefacting produced by DXP PL6.

However, I've now tried processing the Raw file in Topaz, and it gives results very similar to DXP Pl. Both show the same line artefacts on the eye.

 

[Sangoma]

Could we see the original?

 

[Russell Johnson]

I think that globally applying noise reduction is not the most effective way to use it. On an image with significant noise, I would create two DNGs in DXO with different levels of NR and sharpening, one for the background, the other the subject.

You can then blend the two images in PS to provide you with the optimum results on the subject.

 

[myotis]

Could we see the original?

I'm not sure what you mean by the original? The C1 image is as close to the original (.i.e. there is no AI denoising on it, just straight demosaicing) as I can get.

 

[myotis]

I think that globally applying noise reduction is not the most effective way to use it. On an image with significant noise, I would create two DNGs in DXO with different levels of NR and sharpening, one for the background, the other the subject.

You can then blend the two images in PS to provide you with the optimum results on the subject.

This is exactly how I approach denoising, but it doesn't seem relevant to my post.

As I said in the post, I tried different levels of denoising, and models and DXO still produced these line pattern on the eye.

 

[ecoleman]

Do one with LR for the eyes and one with DXO, then blend in PS.

 

[ecoleman]

Not sure if it’s available in DXO PL yes but the latest version of Pure Raw has a force detail slider which could help reduce the lines.

 

[myotis]

Do one with LR for the eyes and one with DXO, then blend in PS.

If I was in need of a solution that would be one way of doing it, but that wasn't the point of the post.

It's only because I processed the files in alternative programs and had an idea of what the eyes looked like in real life that I became aware of how far from reality DXO had rendered them. I was just sharing my observation. I've read several people comment on how DeNoiseAI can cause weird artefacts in feather patterns, but never heard anyone warn about this type of issue with compound eyes.

 

[myotis]

Not sure if it’s available in DXO PL yes but the latest version of Pure Raw has a force detail slider which could help reduce the lines.

Thanks, and It is, and I tried it., It made no difference.

 

[jerry12953]

Not sure if this is relevant but I find DXO Photolab (v5) NR sometimes gives a very pronounced colour cast which I've found difficult to correct. LR AI NR seems pretty consistently correct. I've started to use LR denoising for everything, even though each file takes about 3 minutes on my machine.

 

[myotis]

Not sure if this is relevant but I find DXO Photolab (v5) NR sometimes gives a very pronounced colour cast which I've found difficult to correct. LR AI NR seems pretty consistently correct. I've started to use LR denoising for everything, even though each file takes about 3 minutes on my machine.

There was a problem, at one time, not sure if it was v5 or v6, where DXO was adding the incorrect colour TAG to the exported DNG. It was linked to the new wide gamut colour space they introduced, and was fixed with a point release.

I use DXO PL6 as a "plugin" for C1, and find the colour between the C1 processed NEF and the C1 processed DNG from DXO to be virtually identical.

However, I don't think it's relevant to the way DXO is rendering the detail in the dragonfly eye. I think it;'s just the way DXO is interpreting and "rebuilding" the detail it's finding in the eye.

 

[jerry12953]

Right - maybe I should update my v5 of Photolab. Thanks!

But I agree, my point is not relevant to yours.

 

[LCPete]

Interesting thanks for the heads up
I haven’t noticed any unusual effects from using DXO photolab 5 deep prime raw conversion
I do photograph dragonflies and damselflies in the spring onwards will keep a close watch on what results I get from DXO

 

[myotis]

Interesting thanks for the heads up
I haven’t noticed any unusual effects from using DXO photolab 5 deep prime raw conversion
I do photograph dragonflies and damselflies in the spring onwards will keep a close watch on what results I get from DXO

I hadn't noticed it before either but I'm going keep an eye on it now.

I had noticed weird effects on bird feathers, but with feathers, DXO seemed better than Topaz., which was why settled on DXO rather than Topaz, but Topaz seems a lot better now than it was.

I got a few more hairy dragonfly pictures today (three females all within 100m) so I will have a look tomorrow and see if I get the same effect

 

[LongLensPhotography]

I generally have marginally better results from topaz denoise ai (not photo ai) but it is mostly just not worth it any more over lightroom. If you don't screw up your iso and exposure too badly a very mild 15-20 setting on lightroom is enough. I don't like what it does over 25, and sometimes it creates a lot of green wash. It is then better to do a second pass in denoise ai again at low settings. This would be for extreme example which you just don't get all that often

It takes like 3-5s using 3060ti GPU. Not that bad

 

[myotis]

I generally have marginally better results from topaz denoise ai (not photo ai) but it is mostly just not worth it any more over lightroom. If you don't screw up your iso and exposure too badly a very mild 15-20 setting on lightroom is enough. I don't like what it does over 25, and sometimes it creates a lot of green wash. It is then better to do a second pass in denoise ai again at low settings. This would be for extreme example which you just don't get all that often

It takes like 3-5s using 3060ti GPU. Not that bad

Are you talking about artefacts (which is what this thread is about) or denoising in general?

And which version of PhotoAI? For me PhotoAI has dramatically improved over the last year and I'm no longer using the stand alone version.

But my routine has been to use DXO PL 6 for denoising (but at low settings (luminance at 5-25 rather than the default 40, and lens corrections/sharpening at between 0 and -2, rather than +1). Then refining the sharpening and noise with Topaz PhotoAI (it used to be Topaz DenoiseAI) in a photoshop layer (if needed).

Trying to use DXO on it's own produced artefacts, as did trying to use Topaz on it's own, Using them both at very low settings generally avoided problems, The example I posted is the first real problem I've had since adopting this workflow.

I've also been trying Adobe denoise AI as a replacement for my DXO first pass, but so far I have preferred the results from DXO (most of the time). Not really noise, but I find that even with the sharpening/lens correction at its lowest settings in DXO, any subsequent sharpening in Capture One or PS is more effective/easier than a file denoised with DXO than one denoised with Adobe DenoiseAI. But I'm still experimenting with Adobe DenoiseAI.

I think it depends on what you photograph and what sort of ISOs you routinely use. I see birds and dragonflies as "high quality subjects" where detail and colour are important, but where you are also often forced into very high ISOs. So for me, denoising programs are an important issue, especially as the AI options have always done weird things to feathers.

I gave up entirely on DXO earliest incarnation of AI denoise, because of the artifacting of fine details and textures.

 

[LCPete]

what sort of ISO are you using for Dragonflies Graham ?
I normally on 400 or 800 on Canon R5 and previously 6D and 7D mk 2, have been running them through DXO 5 deep prime with no issues
I haven’t thought of turning down the default luminance from 40 thanks for the heads up on that will experiment with lower settings
I do turn the sharpness settings down though quite a lot the standard default is too high
I have a older version of topaz AI program but dont use it for macro/closeup work
I only use DXO 5 now and Affinity photo

 

[Terrywoodenpic]

It looks like all the programs are interpreting the structural arrangement of the ommatidia as being a moire pattern, and trying to correct it, with various degrees of success. With more learning, Ai should be able to recognise it for what it is and leave the true pattern alone. Of course, moire is an issue with patterns of the size close to that of the sensor. And will also depend on the relative angles of the pattern rotations between them.

It is amazing that Denise programs differentiate between fine dental and noise as well as they do.

 

[myotis]

what sort of ISO are you using for Dragonflies Graham ?
I normally on 400 or 800 on Canon R5 and previously 6D and 7D mk 2, have been running them through DXO 5 deep prime with no issues
I haven’t thought of turning down the default luminance from 40 thanks for the heads up on that will experiment with lower settings
I do turn the sharpness settings down though quite a lot the standard default is too high
I have a older version of topaz AI program but dont use it for macro/closeup work
I only use DXO 5 now and Affinity photo

The picture I posted was 1800 ISO on a Nikon d500. I don't have a "normal" ISO.

I normally have the Nikon memory banks set up to have fast, medium, and slow options. i.e. in terms of the type of action I might want to capture.

All three use aperture priority, but bank B won't allow shutter speeds slower than 1/2500, bank C slower than 1/1250 and bank D slower than 1/800, and each bank has a different limitation on the maximum ISO allowed before longer shutter speeds are made available.

Memory bank A has no specific setting assigned to it, and if things are happening slow enough for me to feel happy with a shutter speed slower than 1/800 I will set my options up from scratch using the memory bank A position

The beauty of the memory banks is that they remember your last settings, as well as providing some restriction on what the camera can do. I find them far more useful than the custom settings on the cheaper Nikons, where the settings are "fixed" for each custom setting.

As I'm doing birds and dragonflies, I tend to walk around on memory bank B, with an aperture that suits the lighting conditions, which is often f7.1 or sometimes wide open, so I can grab a flying bird. Just letting the ISO do its own thing.

With the picture of the Hairy Dragonfly (as posted) I would usually switch to memory bank C, which is normally left at f8, and again let the ISO do its own thing. But I can see from the exposure details that I left it on memory bank B, resulting in an unnecessary high shutter speed, and ISO.

As a principle, now that the denoising software is so good, I would rather risk high ISOs than risk subject or camera movement.

 

[myotis]

It looks like all the programs are interpreting the structural arrangement of the ommatidia as being a moire pattern, and trying to correct it, with various degrees of success. With more learning, Ai should be able to recognise it for what it is and leave the true pattern alone. Of course, moire is an issue with patters of the size close to that of the sensor. And will also depend on the relative angles of the pattern rotations between them.

It is amazing that Denise programs differentiate between fine dental and noise as well as they do.

I agree with the moire idea, but given there is no AI on the C1 image, and that both Adobe AI and Topaz AI are giving results very similar to the C1 interpretation, it seem that DXO is the only one of the three AI programs that needs more learning, in this instance.

But I agree that overall it's amazing what the denoiseAI programs can do, even if they are still inconsistent at times.


The original DXO Prime (that used AI) was pretty awful in this regard as it clearly invented detail that didn't exist. i.e. new tree branches or turning a 4 pane window into one with 6 panes

 

[sk66]

DXO is applying much heavier noise reduction here... I would have expected Topaz Photo AI to do similar; I know it can and does at times. Adobe's denoise/enhance only uses whatever noise reduction you select (last selected). I don't think any of them are misinterpreting the pattern as moire.

 

[jerry12953]

But I agree that overall it's amazing what the denoiseAI programs can do, even if they are still inconsistent at times.

Yes, absolutely! It makes some of my bird photographs look positively pin-sharp.

 

[myotis]

DXO is applying much heavier noise reduction here... I would have expected Topaz Photo AI to do similar; I know it can and does at times. Adobe's denoise/enhance only uses whatever noise reduction you select (last selected). I don't think any of them are misinterpreting the pattern as moire.

I'm not sure I understand the comment on Adobe denoise only using whatever noise reduction you select.m which implies Topaz and DXO are different But both Topaz and DXO allow more manual control over the results than Adobe does

I didn't spend any time trying to match noise reductions, but as I said, I did run through all the options in DXO with both Deep Prime and Deep prime XD. Well at leasts to slightly higher than my original values of Luminance reduction of 20 (compared to the default of 40) and the lens correction/sharpening of -0.5, to the lowest values possible.

All of which produced the same lines on the eye.

I didn't adjust the model strength, which was left at my default of -20.

With adobe I just ran it at 20 and 40.

 

[sk66]

I'm not sure I understand the comment on Adobe denoise only using whatever noise reduction you select.m which implies Topaz and DXO are different But both Topaz and DXO allow more manual control over the results than Adobe does

I didn't spend any time trying to match noise reductions, but as I said, I did run through all the options in DXO with both Deep Prime and Deep prime XD. Well at leasts to slightly higher than my original values of Luminance reduction of 20 (compared to the default of 40) and the lens correction/sharpening of -0.5, to the lowest values possible.

All of which produced the same lines on the eye.

I didn't adjust the model strength, which was left at my default of -20.

With adobe I just ran it at 20 and 40.

I assumed Topaz and DXO were run on some version of Auto or a model (e.g. AI clear)... I've never messed with DXO, and I haven't messed with Topaz much. The only adjustment yio have with Adobe denoise is the amount, but it's not an auto setting.

 

[myotis]

I assumed Topaz and DXO were run on some version of Auto or a model (e.g. AI clear)... I've never messed with DXO, and I haven't messed with Topaz much. The only adjustment yio have with Adobe denoise is the amount, but it's not an auto setting.

Ah, I see.

DXO PL, (and DXO pure raw, but to a lesser extent) and Topaz Photo AI offer far more manual control of the denoising (and detail/sharpening) settings than Adobe DenoiseAI offers.

DXO has default settings and Topaz has an "autopilot" setting, which gives different settings for each file it analyses. But I find these to be far too strong.

I use my own presets for DXO (the reason I bought DXO PL rather than PureRaw), and now Topaz has introduced custom presets in their latest release. This will speed up needing to manually turn down the autopilot suggestions on every file.

 

[myotis]

I've added a couple of new pictures to my original post, which shows that if you start with the RAW file with both DXO and Topaz you get the same artefacting on the eye from both of them.

As you can only use DXO with raw files this means there is no way of avoiding these artefacts with DXO, but there are options to avoid it with LR or C1 with or without Topaz PhotoAI being applied on a processed TIFF.

 

[LongLensPhotography]

And which version of PhotoAI? For me PhotoAI has dramatically improved over the last year and I'm no longer using the stand alone version.

v1 and v2 were dreadful. Full artifacts left and right, top and bottom. I've quickly looked at v3 today and it seems to be comparable to denoise AI, so hardly any point using it; sharpen module still creating nonsense with artifacts all over. Only renewed in FEB when they nearly gave it away, but IMHO still money wasted when they can't even fully match their old apps.

And just to be clear this is with the pre-processed files in LR with Zero sharpness, as recommended. RAW files in Topaz as you have illustrated above is a total and absolute disaster and should not even be in a public release. I realise they will be soon completely sidelined by Adobe, so they are maybe building an all-in-one photo suite from the wrong way round.

5DS files from ISO 1600+ have this really horrid fine grain noise and it really takes two passes: one in LR at 20-25% then Denoise AI / Low Light @ no more than 20 anything to get very clean and believable image. One of them alone just isn't completely enough.
Hopefully this will be a thing of a past when I eventually replace with as currently planned the Z8.

 

[myotis]

v1 and v2 were dreadful. Full artifacts left and right, top and bottom. I've quickly looked at v3 today and it seems to be comparable to denoise AI, so hardly any point using it; sharpen module still creating nonsense with artifacts all over. Only renewed in FEB when they nearly gave it away, but IMHO still money wasted when they can't even fully match their old apps.

And just to be clear this is with the pre-processed files in LR with Zero sharpness, as recommended. RAW files in Topaz as you have illustrated above is a total and absolute disaster and should not even be in a public release. I realise they will be soon completely sidelined by Adobe, so they are maybe building an all-in-one photo suite from the wrong way round.

5DS files from ISO 1600+ have this really horrid fine grain noise and it really takes two passes: one in LR at 20-25% then Denoise AI / Low Light @ no more than 20 anything to get very clean and believable image. One of them alone just isn't completely enough.
Hopefully this will be a thing of a past when I eventually replace with as currently planned the Z8.

I agree with you about the Topaz Raw processing, but even before V3, I found my low level DXO denoising followed by low level tweaking in the Topaz PhotoAI worked well. I'm not sure at what point in the Topaz PhotoAI V2 life cycle I switched from using Topaz DeNoiseAI to Topaz PhotoAI. I also agree that it's a bit difficult to know where Topaz are going, as V3 has added some general processing tools (white balance, exposure ). I assume it's so you can export directly to a Jpeg, without going through another processing program.

I still find the Topaz PhotoAI "autopilot" choices to be unusable.

Since my original post, I have repeated these comparisons on a different Hairy Dragonfly picture, and neither DXO nor Adobe produced the linear striping on the eye, but Topaz still did. However, by adjusting the strength of denoising slider in Topaz, you can switch the striping off and on. But, then I had issues getting enough denoising!!.

It's also erratic, I didn't always get the striping, If I shut the file down and re-open it, so Topaz is forces to re-analyse the file from scratch, I could sometimes avoid the striping.with a fil that had it the last time it was opened !!

I still find PhotoAI a useful toolkit for wildlife photographers, given all it offers, but it takes a bit of effort to get the best out of it, and has its limitations. But there seems to be an update every other week, offering some minor improvements.

I sent Topaz my findings, and got an almost instant reply, expressing their appreciation that I had raised the issue. It wasn't just a boilerplate response as they also commented on the actual pictures I had sent.

I hope your Z8 turns out as good as you hope I'm hoping to also see a Z8,as a replacement for my D500, in the not too distant future

 

[myotis]

As a follow up to this, I have taken some more photographs of a hairy dragonfly, in shade and open soft sunshine. The original was in conditions with a clear sun.

ISO ranged from 12,800 to 720. shutter speeds 1/2500 to 1/1250 and aperture was either f7.1 or f8.

All the photographs were similar to the one below which was at ISO 12,800 and in this instance processed with Adobe DeNoiseAI.



This time, same lens, same camera, but with softer lighting, neither DXO PL6 nor Adobe DeNoiseAI produced the linear artefacts that both DXO and Topaz PhotoAI had produced with the original image. All processed from the RAW.

With Topaz, on this new picture, by adjusting the denoising strength slider, I could switch the linear artefacts off and on depending where I set the slider. But if I got rid of the artefacts, I also struggled to control the noise. With teh previous image regardless of settings I couldn't get rid of the artefacts in Topaz or DXO.

The other aspect with Topaz, was that by simply closing the file down and re-opening it, forcing Topaz's "autopilot" to re-analyse the image, the artefacts were no longer produced. I didn't repeat this often enough to see if there was a clear pattern, but it happened more than once.

I have passed these finding onto Topaz and DXO and both have responded with what is obviously not a boilerplate response, showing genuine interest in looking into the issue. DXO came back asking for additional information about how the photograph was taken.

 

[LCPete]

Really nice shot ,like the way it’s highlighted
I’m jealous they aren’t out yet up here
hope you don’t mind me saying that I just wouldn’t use an ISO anywhere near that high for macro/closeup although I do understand though that you’re experimenting with the raw conversion software
if the light level is low and I don’t have my tripod for something like a dragonfly I would go for a wide enough aperture to get the shutter speed needed and focus stack a few shots and blend them in Affinity photo
realise that you’re experimenting with the software though and didn’t realise that it was possible to get a decent result like that with such a high ISO
very interesting thread thanks for sharing, it’s made me think and experiment with my DXO settings

 

[myotis]

Really nice shot ,like the way it’s highlighted
I’m jealous they aren’t out yet up here
hope you don’t mind me saying that I just wouldn’t use an ISO anywhere near that high for macro/closeup although I do understand though that you’re experimenting with the raw conversion software
if the light level is low and I don’t have my tripod for something like a dragonfly I would go for a wide enough aperture to get the shutter speed needed and focus stack a few shots and blend them in Affinity photo
realise that you’re experimenting with the software though and didn’t realise that it was possible to get a decent result like that with such a high ISO
very interesting thread thanks for sharing, it’s made me think and experiment with my DXO settings

Yes, I should have used a lower ISO, and of the 40 or so pictures I took, most were. around 800 ISO.

The high ISO in the earlier shots was due to how I use the memory banks on my D500, as explained in an earlier post..

My default memory bank B, uses aperture priority and prevents shutter speeds longer than 1/2500, The lens is usually set to f7.1 and the ISO that the camera can choose is unconstrained. This is so I'm set up to grab a shot of flying birds or other active beasties.

With static subjects, I usually grab a couple of shots, in case the beastie flies away, before changing to memory bank C or D, which both have constraints on the ISO range the camera is allowed to use, or I choose entirely manual settings using memory bank A (so I don't mss up the presets in the other memory banks).

Sometimes, I forget to switch memory bank, and I end up with a higher shutter speeds than I need, as well as a higher ISO than I want :-(

 

[LCPete]

Yes, I should have used a lower ISO, and of the 40 or so pictures I took, most were. around 800 ISO.

The high ISO in the earlier shots was due to how I use the memory banks on my D500, as explained in an earlier post..

My default memory bank B, uses aperture priority and prevents shutter speeds longer than 1/2500, The lens is usually set to f7.1 and the ISO that the camera can choose is unconstrained. This is so I'm set up to grab a shot of flying birds or other active beasties.

With static subjects, I usually grab a couple of shots, in case the beastie flies away, before changing to memory bank C or D, which both have constraints on the ISO range the camera is allowed to use, or I choose entirely manual settings using memory bank A (so I don't mss up the presets in the other memory banks).

Sometimes, I forget to switch memory bank, and I end up with a higher shutter speeds than I need, as well as a higher ISO than I want :-(

oh yes I remember you saying
sound like a good way of working, on my Canon I have 3 C settings that can save to memory
1 is for macro and 2 and 3 are for moving subjects and eye detection focus
for macro I normally use ISO 400 or 800 , this time of year I’m normally photographing zoo animals and want to be ready if they move so normally just leave the camera on ISO 1600 which is normally fine