Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Grumpy John
    commented on 's reply
    I'm glad that you're not holding your breath for an answer Alan as I feel you would turn 50 shades of purple before hearing back.
    I am currently in email conversation with DxO about issues that I am having with NIK Collection, so far it has been 7 weeks and the problem has still not been resolved. They have made plenty of suggestions, but nothing has worked.

  • wigz
    commented on 's reply
    Phil,

    It's not Microsoft software but distributed in their store, which is their version of the Apple and Google stores.

    There is some proper software there but also plenty of minimally functional stuff.

    About the cleanup - yep, would be nice but I'm not holding my breath.

  • Ozzie_Traveller
    commented on 's reply
    G'day fellas
    GJ you said above ...

    I'm also trying out a program from the Microsoft page called Duplicate & Similar Photo Cleaner, although it loads up as Photo Forensics Free (go figure). I'll let you know how it goes.

    This is typical MS behaviour -- buy up an existing bit of "pretty good" software, reverse engineer it into Windows and then put it out as a MS creation

    Alan- your efforts are beyond the realm of "good works for the family" ... I hope they take you out to a damn fing eatery once it's all finished

    Phil

  • wigz
    commented on 's reply
    No John. The email bounced - mailbox full - and went to the spam folder so I didn’t see it for a while. I’ll try again but I’m not hopeful.

  • Grumpy John
    replied
    Hi wigz have you heard back from the developers of Duplicate and Similar Photo Cleaner yet, if so did they have any clues on improving performance?

    Leave a comment:


  • wigz
    commented on 's reply
    I think I have found the problem. The EXIF orientation flag is different in the identical images that don't match. The following from ExifTool output:

    Image 1: Orientation : Rotate 90 CW
    Image Size : 3264x2448

    Image 2: Orientation : Horizontal (normal)
    Image Size : 2448x3264

    I suspect both programs are not handling this correctly. I have no idea why this flag is different in these images - remember these are not mine.
    Back to manual checking and deletion - at least there aren't as many now,

    Below are some tests I ran to try to determine what was going on,

    I tried VisiPics on all photos and it picked up about 60 that Duplicate and Similar Photo Cleaner (DASPC) did not find. These seemed to be where there was a drastic size difference such as 50k to 3000k.

    There are many images that both programs missed. In one folder of about 230 images there were about 100 duplicates with minor differences in file size but the images were dimensionally and visually identical, 2.3MB to 2.4MB for example. Both programs missed these.

    Using DASPC on this folder at the lowest similarity setting, and highest CPU setting found one pair of visually similar images that were in fact similar, not duplicates, but missed all of the duplicates..

    Using ViisiPics at Basic similarity setting found 19 pairs of similar images that were atually duplicates, but showed one of them rotated ninety degrees - both thumbnail and preview. All other programs showed these oriented correctly. I had the option 'Scan for 90 degree rotations set.

    I took two identical photos that were not being picked, opened them in photoshop, drew a small spot on each and saved as a different name, and checked with each program again.

    Both programs identified three images as the identical, both modified versions and the other identical image that I had not touched, but missed the original unmodified image.

    VisiPics has a number of usability issues that make it more awkward to use than DASPC.

  • wigz
    commented on 's reply
    Haven't tried VisiPics as 'Duplicate and Similar...' seemed good until I was most of the way through. I might give it a go to see if it picks up any others.

    I haven't found many videos it missed so far. I'm still doing a final review and clean-up.

  • Grumpy John
    commented on 's reply
    As a comparison, have you tried VisiPics?
    How did it go with the video files?

  • wigz
    commented on 's reply
    One big scan John. I included videos and it did pick up a lot of them. I have also found a number of other duplicate jpegs it missed for which I can see no reason.

    I moved four images that it had picked up as two sets of two duplicates into a separate folder and just ran a scan of that folder. It gave the same result.

    Interestingly I had it scan those four for similar, rather than duplicates, and it found none. It obviously has some issues but did find a lot more duplicates than I would have found without it, and in less time.

  • Grumpy John
    commented on 's reply
    Interesting read Alan. Did you do one big scan, or did you divvy it up into groups of smaller scans? Also did you try a search for duplicate videos?

  • wigz
    commented on 's reply
    If anyone is interested, I finished using the 'Duplicate and Similar Photo Cleaner' program to delete duplicates and found a couple of issues. It does seem to handle rotated images but does miss duplicates as described in the below extract from the email I sent to the developers.

    I was generally impressed by the program and consider it a bargain, but found a couple of issues.

    The program did not identify a number of images as duplicates even though they were as I verified it using 'Differences' layer blending in Photoshop. In this case there were four identical images that the software matched as two sets of two (i.e. each with one duplicate as the program counts). There were many other examples with groups of more images.

    The only difference that I could see was that the non-matched images had a different value in the Application tag in the EXIF information. e.g:
    (<xmp:CreatorTool>Microsoft Photo Gallery 16.4.3528.331</xmp:CreatorTool>

    I have attached two images that should be identified as duplicates as well as a screen-shot of the program's match display.

    Also the log contained a large number of error messages like this example:
    Could not recycle file:G:\alwigginton\Pictures\GraemePC\Pictures\201 7\03\629.jpg - Could not find file 'G:\alwigginton\Pictures\GraemePC\Pictures\2017\03 \629.jpg'

    I did not use automatic deletion for most images but deleted them manually using the program's interface. It is possible that I may have deleted several outside the program, but definitely not this many.

    Most, but no all, of these messages came from the automatic deletion I used for the last of the images - those with a single match. Is it possible that the same photo appeared in several match groups, so the second would cause the 'not found' error.

    At least one of the files listed as not found is in the recycle bin from an earlier deletion.

    A couple of suggestions for improvements that you might like to consider:
    - Update the count of duplicates at the bottom of the match page as images are removed
    - Allow searching of the match list for a file name, preferably a partial match. I would have found this very useful when verifying the programs matching.
    - Provide a button to delete the images marked for deletion from the current group. I had a situation where I often wanted to keep a different version to what the program recommended. I guess I could have gone through and changed the flags and then done the full auto delete,

    Thanks for a useful program.

  • wigz
    commented on 's reply
    Me too. The paid version is running now. It takes a few hours to process this set of images. I'm only scanning for duplicates rather than similar, as there are many different but similar images and I'm not sure exactly what 'similar' means. It will certainly save lots of time.

    I was a bit concerned that I bought the new version before deleting the old, forgetting that the Microsoft store downloads and installs automatically, but the free and paid versions are named differently - in the start menu at least.

  • Grumpy John
    commented on 's reply
    You're welcome Alan.
    I actually paid for this program this morning because I was so impressed with it. The bonus for me was that it was able to analyse DNG images which VisiPics didn't do.
    If you get the paid version you will need to delete the free version.

  • wigz
    commented on 's reply
    I ran the Duplicate and Similar Photo Cleaner (what an awkward name) and it appeared to work impressively well. I found a couple of issues:
    - It doesn't handle rotated images
    - There is no way to sort the list of matches
    - There is no way to find an image in the list of groups or matches.

    I haven't found any false matches yet but I don't quite trust it enough to let it delete the duplicates automatically yet.

    Unless I find some other issues I will pay for the full version.

    I wasn't impressed the last time I looked at an image matching program (probably years ago) but this looks good.

    Thanks again John.

  • Ozzie_Traveller
    replied
    G'day fellas

    Good info & experiences here .............. and I'm sure we've all done tasks like this and with varying results
    Phil

    Leave a comment:

Working...
X