Question about 'photoshop' case.

[zero-suger]

Thank you such a hard work. I really interested in this project and 'DeCLIP' model. But when I tested with 'photoshop' edited photos (nose, eyes swapped) model performance is very poor. I tested with each checkpoint. Have you ever consider this case? If yes, what I have to do consider? DeCLIP possibly can catch 'photoshop' edited images?

Thank you

[elimarinoiu]

Hi,

Thank you for your message and for using DeClip. We have not tested with photoshop edited images.
Can you give us more details on your results (when testing with the LDM checkpoint)?

• Have you evaluated the performance numerically (with what metric and threshold)?
• At a visual inspection, how does the model fail:
  - it does not detect the manipulated area at all
  - the manipulated area is detected but the signal is weak ( below 0.5)
  - other behaviour?
  Best,
  Elisabeta

[zero-suger]

Hi,

Thank you for your message and for using DeClip. We have not tested with photoshop edited images. Can you give us more details on your results (when testing with the LDM checkpoint)?

• Have you evaluated the performance numerically (with what metric and threshold)?
• At a visual inspection, how does the model fail:
  • it does not detect the manipulated area at all
  • the manipulated area is detected but the signal is weak ( below 0.5)
  • other behaviour?
    Best,
    Elisabeta

Hi ~

I tried to do these steps :

1. Capture 4 difference ID photos with S24 Ultra, and swap only 'Eyes and the nose' in each ID photos.
2. Detect face with MTCNN and crop each face with 0.005% extra margin around the face and then test the DeCLIP model localization and detection code. And I am sending the results pic.
   

Please could you help me to solve this problem? Am I doing somthing wrong in image preprocessing?

Thank you,

Aziz

[elimarinoiu]

Hi Aziz,

DeCLIP was trained and tested on images modified by fully generating a particular face area. This process leaves traces that can potentially be picked up by DeCLIP. The images you are testing on seem to be using swapping or splicing of real content on another image which does not involve generated content. You could try to train DeCLIP using the manipulations you wish to detect.

Best,
Elisabeta