Dr. Michael S. Lew

Director, Deep Learning and Multimedia Retrieval Research Group
co-Head, Computer Systems, Imagery and Media Research Cluster
Computer Science Department (LIACS)
Leiden University



My Positions and Research

  • Co-head of the LIACS Computer Systems, Imagery and Media Research cluster (30+ members), which is one of the two main research groups at the computer science department at Leiden University.

  • Chair Full Professor at Tsinghua University, 2003-

  • Tenured associate professor at Leiden University, 2001-

    Research - short version

    It is currently impossible to find most scientific, personal and social imagery because meaningful annotations describing the content are missing. My work will allow all imagery to be searched by developing new deep learning algorithms for automatic image annotation and by exploiting synergy between the user and the computer in novel interactive search paradigms.

    Research - longer version

    With the ubiquity of the Internet, social and streaming imagery and media, there has been an explosion of information that is now accessible to the entire world. The importance of the multimedia information can not be underestimated. It is a part of our daily lives and shapes how we experience and perceive and interact with the world. Other periods in history were dominated by materials for society survival (warfare), sustenance and growth. Surely, this time we live in is the "Age of Information." In particular, multimedia information is the fundamental pillar for shaping modern societies from personal events to elections of leaders to international news and politics and even waging conflict. One prominent characteristic of the mountain of multimedia information is that while it is indeed accessible, it is extremely difficult to search for a desired item - a needle in a mountain of haystacks. My group's mission directly addresses this problem. We investigate new paradigms for the computational retrieval and understanding of multimedia information by developing new deep learning approaches. Currently, keyword search (such as Google and Bing) is the dominant method for searching for information in large datasets. This works well for text documents, but the problem is that most of the images and video worldwide do not have annotation - e.g. the vast number of smartphone pictures typically only have date and location, not who or what was in the picture. In this case the only option is to analyze the pictorial content of the images and video toward extracting keywords and then using those keywords for the search queries.

    Teaching/Courses

    LIACS News Research Grants (either leading or co-investigator as of June 2017; my part of the grant below)
    • Computer Aided Medical Diagnosis Using Big Data - NVIDIA, CSC, 1 PhD

    • Data Mining on High Volume Simulation Output - NWO, 1 PhD

    • Deep Visual Understanding of Paintings and Pictures - NVIDIA, CSC, 1 PhD

    • Making Sense of Illustrated Handwritten Archives - NWO, 1 PhD

    • Transmedia storytelling voor kritisch engagement - NWO, 1 assistant/associate professor

    Influential People Activities (selected) University & Department Positions
    • co-Head of the Imagery and Media Research Cluster - One of the 4 research themes in the Computer Science (CS) dept.

    • Chair, LIACS MSc Education Committee

    • Member (and 2nd chair) of the Board of Examiners (ExamenCommissie) - oversees the quality of all CS dept. courses (until 2014)

    • Member of the Curriculum Committee - guides the direction and content of the CS dept. courses

    • Member of the Organizing Team for the LIACS Research Seminar

    • co-Director of the LIACS Media Lab - advises and guides research and teaching in multimedia technology within the CS dept.

    • Member of the LIACS Scientific Council - steering committee for all research in the CS dept.
    Deep Learning Publications - Recent (At this weblink are some tips for starting points and a nice taxonomy is at CNN Taxonomy)
    • Bag of Surrogate Parts Feature for Visual Recognition, IEEE Transactions on Multimedia, volume 20: 1525-1536, 2018

    • Learning visual and textual representations for multimodal matching and classification, Pattern Recognition, volume 84: 51-67, 2018

    • A comprehensive evaluation of local detectors and descriptors, Signal Processing: Image Communication, Volume 59, November Pages 150-167, 2017

    • Deep Binary Codes for Large Scale Image Retrieval, Neurocomputing, 2017

    • Deep learning for visual understanding, Neurocomputing, 2016

    • Learning a Recurrent Residual Fusion Network for Multimodal Matching, ICCV - IEEE Int. Conf. Computer Vision, 2017

    • On the Exploration of Convolutional Fusion Networks for Visual Recognition, MMM - International Conference on MultiMedia Modeling, 2017, best paper award

    • Improving the Discrimination between Foreground and Background for Semantic Segmentation, ICIP - IEEE Int. Conf. Image Processing, 2017

    • Learning Relaxed Deep Supervision for Better Edge Detection, CVPR - IEEE Conf. Computer Vision and Pattern Recognition, 2016

    • Bag of Surrogate Parts: one inherent feature of deep CNNs, BMVC - British Machine Vision Conference, 2016

    • DeepIndex for Accurate and Efficient Image Retrieval, ACM ICMR - Int. Conf. Multimedia Retrieval, 2015

    Editorships Books

    Conference Organization

    Scientific Conference Program Committees

    Representative Publications (over 100 peer-reviewed in ACM, IEEE, and LNCS)
         Browse LIACS publications

    Recent Department Activities

       (1) 4 Semester length courses, 24 ECs (Jan - Dec 2010)
       (2) LIACS Media Lab for education/research (hardware/software facilities, knowledge)
           - Guidance, programming expertise and system administration for at least 5 courses & 30 students per year
       (3) LIACS/Mathematics/I-Group Information Screens & Website
       (4) LIACS Research Seminar (rebooted 2011)
       (5) Eureka Paper (at request of Educational Director)
       (6) Open Day Activities - Talks & Demonstrations
       (7) Science Based Business collaboration - video interviews
       (8) Exam Committee
       (9) Curriculum Committee
       (10) Public Relations Committee
           - Social Media Initiative (i.e. YouTube)
           - Google results initiative

    Recent Graduate Students

    • Nan Pu
    • Wei Chen
    • Wouter B. Verschoof-van der Vaart, DSRP Grant
    • Theodoris Georgiou, NWO Grant
    • Yu Liu (graduating with Ph.D. on October 24th, 2018)
    • Yanming Guo (graduated with Ph.D. on October 5th, 2017)
    • Song Wu (graduated with Ph.D. on December 22nd, 2016)
    • Susan Laraghy
    • Zhenyang Li
    • Ran Tao
    • Simon Zaaijer
    • Ard Oerlemans (graduated with Ph.D. on December 22nd, 2011)
    • Bart Thomee (graduated with Ph.D. on November 3rd, 2010)
    • Nicu Sebe (graduated with Ph.D. on March 28th, 2001)

    Interesting Projects

    Old Research Projects (funded)
    • CSC - Multimedia Information Retrieval
    • CSC - Visual Learning Using Big Data
    • BRICKS Project (member of the project board) - Interactive Search and Browsing
    • Cyttron (co-leader of the BioSearch Project) - Multi-modal imaging search
    • Advanced Information Processing in Bioinformatics, NBIC Biorange IV - VL-e (investigator)
    • VIRSI Project (project leader) - Computational Imagination for Intelligent Search
    • Philips Research Project on Bio-Medical Analysis (project leader)
    • Philips Research Project on Texture Analysis (project leader)
    Multimedia Links Class Schedules

    Other Activities: Student Advising and Research Grants (Click Here)

    Contact information

    Main Contact:

    Telephone: 31-71-527-7034

    Fax: 31-71-527-6985

    Postal Address:

        Leiden Institute of Advanced Computer Science
        Leiden University
        Niels Bohrweg 1
        2333 CA Leiden
        The Netherlands
    


    Publications (deep learning and multimedia retrieval)

    Deep learning publications and demo


    Publications (technical reports, preliminary work)

    Content-based tag recommendation algorithms for unstructured data

    Improving SIFT accuracy with use of perspective transforms

    Improving the LSDh-tree for fast approximate nearest neighbor search

    Information-Synthesis Network for Facial Landmarks Estimation

    Sub-Image Search Engine

    Image Similarity Using Color Histograms

    Rating Inference


    PhD at Leiden University

    PhD at Leiden Guidelines

    PhD Training and Compulsory Courses

    ASCI Research School

    CSC and Leiden

    PhD Defense Rules and Guidelines - Cache


    Links

    Bus - Corpus

    Bus - Niels Bohrweg

    Bus - Universiteitsterrein

    Bus - De Kempenaerstraat

    Google.com - A good WWW search engine for general information and guides

    Google Search Operators

    Bing Search Operators (e.g. inbody:)

    Thesis Check - Turnitin

    IBM Watson Speech to Text

    Bus Corpus

    Bus Niels Bohrweg

    Dutch Basisverzekering

    Doctor Boender

    DBLP - Michael S. Lew

    DBLP - International Journal of Multimedia Information Retrieval

    MarBo Young on Writing

    Log out Amazon.com

    Log out Amazon.co.uk

    Archive.org - Way Back Machine (historical WWW)

    SAP Leiden

    LIM

    Waist to Height - Penn Health

    Sunblock Ingredients

    Random links

    Campaign Promises - BBC

    Campaign Promises - USA Today

    Campaign Promises - Github

    Six Party Talks and Signed Declaration 2005

    GOPDND

    Dilbert

    Perldoc.com - Perl Documentation

    Multimedia Conferences 2011

    AMSR links

    ACM International Conference on Multimedia Retrieval

    ACM International Conference on Multimedia Retrieval, Glasgow, 2014 Web Archive

    ACM TOMCCAP MIR Survey (vol. 2, issue 1, pp. 1-19, 2006)

    Content-based tag recommendation algorithms for unstructured data

    Improving SIFT accuracy with use of perspective transforms

    Improving the LSDh-tree for fast approximate nearest neighbor search

    Information-Synthesis Network for Facial Landmarks Estimation

    Sub-Image Search Engine

    Image Similarity Using Color Histograms

    Rating Inference