Ars Electronica Archive
Prix
Prix Map
Pictures
Videos
Publications
Special Projects
Women in Media Art
Prix
The Prix Ars Electronica Archive is a collection enabling search and viewing of all the submissions since 1987. The award-winning projects are documented with catalogue texts and audio-visual media. All other submissions can be searched by title/artist and displayed with year, category in list form. Please cite the credits (artwork name, artist and photographer) and only use the materials if your article is related to Ars Electronica.
VFRAME: Visual Forensics and Metadata Extraction
Adam Harvey
Original: feature.jpg | 1920 * 1080px | 315.8 KB
Original: vframe_synthetic_data_colormap.jpg | 1920 * 1080px | 200.8 KB
Original: vframe_ao25_wireframe_photo.jpg | 1920 * 1080px | 146.4 KB
Original: anim_overhead.gif | 960 * 540px | 956.2 KB
Original: ao25_wireframe.jpg | 960 * 540px | 91.4 KB
Original: VFRAME | 3840 * 2166px | 2.7 MB
Original: VFRAME | 7232 * 4068px | 29.9 MB
Original: VFRAME | 3840 * 2160px | 23.8 MB
*VFRAME* is a research project exploring how computer vision can be applied to human rights research. *VFRAME* started in 2017 as a prototype demonstrating how object detection can be used to locate illegal munitions in the Syrian conflict. During 2018, *VFRAME* worked with the Syrian Archive (*syrianarchive.org*) to develop a large-scale visual search engine and custom algorithms for processing million-scale video datasets. Then, a new problem emerged: it was only possible to create object detection models for objects with thousands of examples, yet most objects in conflict zones only appear in a limited number of videos, often under extreme circumstances and with low-resolution cameras. To train new object detection models relevant for human rights researchers working with footage from conflict zones a new approach was needed.
During 2019 *VFRAME* is developing synthetic data modeling techniques that can be used to create specialized datasets. Synthetic training data is 3D-modeled photorealistic imagery based on a set of real, verified images. Often only a few real images are needed to recreate the object as a 3D model. The 3D model can then be textured and placed into a 3D environment to match the real images. This 3D environment can then be customized to generate lighting, weather, camera, and scene variations that are exported to still frames and used as image training data.
The first prototype shows how this method can be applied to an illegal cluster munition, the AO-2.5RT. Although 3D modeling the objects and scenes is still a labor-intensive process, it has the potential to be scalable and provide training data for objects that would otherwise not exist in the quantity and quality needed to train object detection models. This new approach has the potential to vastly increase the potential applications of computer vision for human rights researchers and will be carried out through a collaboration with the Syrian and Yemeni Archive projects based in Berlin.
Links: https://vframe.io/research/synthetic-datasets/
During 2019 *VFRAME* is developing synthetic data modeling techniques that can be used to create specialized datasets. Synthetic training data is 3D-modeled photorealistic imagery based on a set of real, verified images. Often only a few real images are needed to recreate the object as a 3D model. The 3D model can then be textured and placed into a 3D environment to match the real images. This 3D environment can then be customized to generate lighting, weather, camera, and scene variations that are exported to still frames and used as image training data.
The first prototype shows how this method can be applied to an illegal cluster munition, the AO-2.5RT. Although 3D modeling the objects and scenes is still a labor-intensive process, it has the potential to be scalable and provide training data for objects that would otherwise not exist in the quantity and quality needed to train object detection models. This new approach has the potential to vastly increase the potential applications of computer vision for human rights researchers and will be carried out through a collaboration with the Syrian and Yemeni Archive projects based in Berlin.
Links: https://vframe.io/research/synthetic-datasets/
Founder: Adam Harvey
Co-developer: Jules LaPlace
3D designer: Josh Labouve
Researchers: Hadi Al-Khatib and Jeff Deutch at Syrian Archive
Support received from: PrototypeFund.de
Co-developer: Jules LaPlace
3D designer: Josh Labouve
Researchers: Hadi Al-Khatib and Jeff Deutch at Syrian Archive
Support received from: PrototypeFund.de
Adam Harvey (US) is an American artist and researcher based in Berlin, who focuses on computer vision, privacy, and surveillance technologies. He is a graduate of New York University's Interactive Telecommunications Program and previously studied engineering and photojournalism at the Pennsylvania State University. Harvey's previous work includes *[CV Dazzle](https://ahprojects.com/cvdazzle/ "CV Dazzle")*, the *[Anti-Drone Burqa](https://ahprojects.com/stealth-wear/ "Anti-Drone Burga")*, *[SkyLift](https://github.com/adamhrv/skylift "SkyLift")*, and *[MegaPixels](https://megapixels.cc/ "MegaPixels")*. He is currently a researcher in residence at Karlsruhe HfG with a focus on machine learning image datasets.