MediFor - Media Forensics

Project executive summary

The FENCE - the subproject lead by University of Florence and included in the MediFor project framework - aims at designing a set of reliable forensic tools based on the visual analysis of image physical properties. These tools will detect and locate several kinds of image manipulations working in a completely automatic way.

The core technical challenge of FENCE is to build a computational architecture capable of assessing the physical integrity of an image included in a very large dataset, with no manual input or any other a priori information about its visual content. This process involves the automatic detection/localization of scene level characteristics, that typically require the human assistance.

The above challenge can be addressed through a sequence of three intermediate goals: first, a general, modular framework (GOAL 1) to build and evaluate vision-based forensic approaches for physical integrity assessment will be defined. Then, based on the proposed framework, modules that will exploit specific image physical properties for detecting important types of image forgeries will be designed and implemented (GOAL 2). Finally, the implemented modules will be refined within an attacker-aware scenario, thus improving both their effectiveness and robustness (GOAL 3).

The main drawback of state-of-the-art physical integrity based solutions is that it is difficult to apply them in an application scenario like the one contemplated in MediFor, requiring that huge amounts of visual data be automatically processed. As a matter of fact, all the available forensic solutions typically require the interaction of a user that manually selects some image characteristics to be exploited for integrity assessment. Moreover, until today nobody has been able to objectively evaluate the performance of forensics solutions, partly due to the impossibility to test human assisted procedures on massive amounts of data, and partly given the unpredictability of human annotation accuracy.

The main novelties of our work are intertwined with the project goals described above. First, the new framework will provide a solid methodology for the definition of novel forensic modules, with clear computational procedures and assessment protocols. Second, the proposed modules will exploit advanced computer vision technologies to avoid any manual intervention during the extraction of image characteristics and their subsequent analysis. This will be a major improvement w.r.t. the state-of-the-art in digital forensics. Third, the inclusion of counter forensic methods into the design loop will provide us with an in-deep understanding of the critical points of each module and suggest a way to overcome them.

If successful, FENCE will allow the application of forensic procedures to large volumes of visual data, thus making possible the timely and automatic exposure of forgeries, thus helping governments and international organizations to better contrast digital crimes and contribute to a safer society.