Conference Proceedings


Alan F. Smeaton


Computer Science

multimedia systems machine learning object recognition activity recognition social media support vector machines object detection neural networks

Generative forms of multimedia content. (Opening keynote talk) (2020)

Abstract Multimedia analysis has recently made spectacular improvements in both quality and in sophistication. Over the last half-decade we have seen extreme progress in tasks like image and video tagging, object detection and activity recognition, generating descriptive captions and more. Some of these have been deployed and are in widespread use in our smartphones and on our social media platforms. We have also seen recent research work, including our own, on computing more abstract features of multimedia, such as person-counting from CCTV, computing visual salience, estimating aesthetics of images and videos, and computing video memorability. The common methodology used across most of these applications is of course machine learning, in all its forms, from convolutional neural networks to simple regression and support vector machines. Much of the research in our field is about wrestling with machine learning to optimise its performance in multimedia analysis tasks and this recent run of extreme progress does not look like ending anytime soon, though it will reach its high water mark. When it does reach the point at which it cannot get any better, what then ? Generative machine learning (ML) is a recent form of media analysis which turns the conventional approach on its head and its methodology is to train a model and then generate new data. Example applications of generative ML deoldify which colourises black and white images and video clips, and Generative Adversarial Networks (GANs) which can generate DNA sequences, 3D models of replacement teeth, impressionist paintings, and of course video clips, some known as deepfakes. Putting aside the more nefarious applications of deepfakes, what is the potential for generative forms of multimedia ? In the short to medium term we can speculate that it would include things like movie augmentation but it how far can it go and could it replicate human creativity ? In this talk I will introduce some of the recent forms of generative multimedia and discuss how far I believe we could go with this exciting new technology.
Collections Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Multimedia systems
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: INSIGHT Centre for Data Analytics

Full list of authors on original publication

Alan F. Smeaton

Experts in our system

Alan F. Smeaton
Dublin City University
Total Publications: 492