Prof. dr. Michael S. Lew

Professor, Hoogleraar Deep Learning
Head, Deep Learning Research Group
Head, Machine Learning Cluster (30+ members)
co-Director, LIACS Media Lab
co-Head, Computational Imaging and Deep Learning Research Group
Computer Science Department (LIACS)
Leiden University
lewmsk@gmail.com




My Positions, Education and Research

  • Full Professor, Hoogleraar Deep Learning at LIACS, Leiden University

  • Head, Machine Learning Cluster (30+ members)

  • Co-head of the LIACS Computer Systems, Imagery and Media Research cluster (70+ members), which is one of the two main research divisions at the computer science department at Leiden University.

  • PhD, University of Illinois Urbana-Champaign (UIUC)

    Research - short version

    My research lies in the field of artificial intelligence with a primary focus on the area of deep learning. I develop new deep learning algorithms toward understanding large databases and libraries of information by learning synergistic methods using multiple modalities within multimedia including but not limited to vision, audio and text. The goal is to bring to light all of the information in the world and at the same time automatically be able to detect misinformation (e.g. fake news).

    Research - longer version

    My research is toward developing artificial intelligence (AI) and currently focuses on the paradigm of deep learning.

    Artificial Intelligence is the most promising paradigm to help society. Major breakthroughs are expected in the near future in a numerous important areas of society (e.g. medicine, automatic self-driving vehicles, fake news detection, robotics, etc.) but only with access to very large datasets. In many areas, the datasets exist but scientists do not have access. An example I am often asked by medical researchers: what it would take to have expert level classifiers for well known diseases and cancers. My reply is usually that one needs access to a very large dataset of examples. These datasets exist already at many research hospitals around the world. However, the datasets can not be used for training the deep classifiers due to privacy reasons. In my opinion, while it is important to develop AI to detect diseases and cancers, it is even more important to protect human privacy rights. Currently, there is no easy answer for the dataset access issue and each country is wrestling with the tradeoff of privacy vs scientific advances in their own way.

    My current research is exploring how to both find information and also how to detect false information. Both are necessary to a modern information retrieval system. Because humans want to express their search queries using high level human concepts, the grand challenge in multimedia retrieval has been "bridging" the semantic gap which is the gap between the high level concepts of humans and the low level features from multimedia. For a specific example, the problem of image classification is especially important because it can bridge the semantic gap by taking an input image and outputting human level concepts. If we can solve the semantic gap, then it would make all of the collected art/heritage, scientific, WWW and personal images searchable and accessible. Furthermore, there is major interest in explanation based modeling which seeks to give better understanding of the underlying reasons and causes involved in deep network decisions.

    Misinformation is one of the key 21st century problems and manifests in many different ways. "Fake News" is perhaps the most well known type and has affected areas from medicine (dangerous treatments for COVID) to politics (unfounded claims of election fraud). Misinformation also occurs in more subtle ways such as intentionally misrepresenting someone's viewpoints such as taking a small part of a discussion and presenting it as the main viewpoint. The context is important. Other modern ways in which misinformation is happening is with misrepresentation of authorship (see ChatGPT) and plagiarism. Part of my current research examines methods of detecting fake information using deep learning.

    Teaching/Courses

    LIACS News Research Grants (ongoing as of December 2020)
    • Computer Aided Medical Diagnosis Using Big Data - NVIDIA, CSC, 1 PhD

    • Data Mining on High Volume Simulation Output - NWO, 3 PhDs

    • Deep Visual Understanding of Paintings and Pictures - NVIDIA, CSC, 1 PhD

    • Making Sense of Illustrated Handwritten Archives - NWO, 2 PhDs, 1 postdoc

    Influential People Activities (selected) University & Department Positions
    • co-Head of the Imagery and Media Research Cluster - One of the 4 research themes in the Computer Science (CS) dept.

    • Chair, LIACS MSc Education Committee

    • Member (and 2nd chair) of the Board of Examiners (ExamenCommissie) - oversees the quality of all CS dept. courses (until 2014)

    • Member of the Curriculum Committee - guides the direction and content of the CS dept. courses

    • Member of the Organizing Team for the LIACS Research Seminar

    • co-Director of the LIACS Media Lab - advises and guides research and teaching in multimedia technology within the CS dept.

    • Member of the LIACS Scientific Council - steering committee for all research in the CS dept.
    Deep Learning Publications - Recent (At this weblink are some tips for starting points and a nice taxonomy is at CNN Taxonomy)
    • A memorizing and generalizing framework for lifelong person re-identification, IEEE TPAMI, 2023

    • Deep Learning for Instance Retrieval, IEEE TPAMI, 2022

    • COCA: COllaborative CAsual Regularization for Audio-Visual Question Answering, AAAI, 2023

    • Lifelong Fine-grained Image Retrieval, IEEE Transactions on Multimedia, 2022

    • Feature Estimations based Correlation Distillation for Incremental Image Retrieval, IEEE Transactions on Multimedia, 2021

    • Lifelong Person Re-Identification via Adaptive Knowledge Accumulation, CVPR - IEEE Conf. Computer Vision and Pattern Recognition, 2021

    • Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval, Pattern Recognition, 2021

    • Multi-Stage Hybrid Embedding Fusion Network for Visual Question Answering, Neurocomputing, Volume 423, 29 January 2021, Pages 541-550

    • New Ideas and Trends in Deep Multimodal Content Understanding: a review, Neurocomputing, Volume 426, 22 February 2021, Pages 195-215

    • Dual Gaussian-based Variational Subspace Disentanglement for Visible-Infrared Person Re-Identification , ACM MM - Int. Conf. Multimedia, 2020

    • SWAPGAN: A Multistage Generative Approach for Person-to-Person Fashion Style Transfer, IEEE Transactions on Multimedia, 2019

    • Bag of Surrogate Parts Feature for Visual Recognition, IEEE Transactions on Multimedia, volume 20: 1525-1536, 2018

    • CycleMatch: A Cycle-consistent Embedding Network for Image-Text Matching, Pattern Recognition, volume 93, 2019

    • Learning visual and textual representations for multimodal matching and classification, Pattern Recognition, volume 84: 51-67, 2018

    • A comprehensive evaluation of local detectors and descriptors, Signal Processing: Image Communication, Volume 59, November Pages 150-167, 2017

    • Deep Binary Codes for Large Scale Image Retrieval, Neurocomputing, 2017

    • Deep learning for visual understanding, Neurocomputing, 2016

    • Learning a Recurrent Residual Fusion Network for Multimodal Matching, ICCV - IEEE Int. Conf. Computer Vision, 2017

    • On the Exploration of Convolutional Fusion Networks for Visual Recognition, MMM -- International Conference on MultiMedia Modeling, 2017, best paper award

    • Improving the Discrimination between Foreground and Background for Semantic Segmentation, ICIP -- IEEE Int. Conf. Image Processing, 2017

    • Learning Relaxed Deep Supervision for Better Edge Detection, CVPR - IEEE Conf. Computer Vision and Pattern Recognition, 2016

    • Bag of Surrogate Parts: one inherent feature of deep CNNs, BMVC - British Machine Vision Conference, 2016

    • DeepIndex for Accurate and Efficient Image Retrieval, ACM ICMR -- Int. Conf. Multimedia Retrieval, 2015

    Editorships Books

    Conference Organization

    Scientific Conference Program Committees

    Representative Publications (over 180 peer-reviewed in ACM, IEEE, and LNCS)
         Browse LIACS publications

    Selected Past Research Milestones

        - My first IEEE Trans PAMI article (in the special issue on Learning in Computer Vision), 1994

        - My first position as Chair Full Professor at Tsinghua (first contract), 2003

        - My first General co-Chair service for the International Conference on Image and Video Retrieval (CIVR), 2003

    Recent Graduate Students (2016-)

    • Kai He
    • Mingrui Lao
    • Nan Pu
    • Wei Chen (graduated with Ph.D. on October 13th, 2021)
    • Theodoris Georgiou (graduated with Ph.D. on September 29, 2021)
    • Yu Liu (graduating with Ph.D. on October 24th, 2018) - Associate Professor, School of Information Science and Engineering, Dalian University of Technology, China
    • Yanming Guo (graduated with Ph.D. on October 5th, 2017) - Associate Professor, College of Systems Engineering, National University of Defense Technology, China
    • Song Wu (graduated with Ph.D. on December 22nd, 2016) - Associate Professor, Southwest University, China

    Interesting Projects

    Old Research Projects (funded)
    • CSC - Multimedia Information Retrieval
    • CSC - Visual Learning Using Big Data
    • BRICKS Project (member of the project board) - Interactive Search and Browsing
    • Cyttron (co-leader of the BioSearch Project) - Multi-modal imaging search
    • Advanced Information Processing in Bioinformatics, NBIC Biorange IV - VL-e (investigator)
    • VIRSI Project (project leader) - Computational Imagination for Intelligent Search
    • Philips Research Project on Bio-Medical Analysis (project leader)
    • Philips Research Project on Texture Analysis (project leader)
    Research (Multimedia) Links Class Schedules

    Other Activities: Student Advising and Research Grants (Click Here)

    Contact information

    Main Contact:

    Telephone: 31-71-527-7034

    Fax: 31-71-527-6985

    Postal Address:

        Leiden Institute of Advanced Computer Science
        Leiden University
        Niels Bohrweg 1
        2333 CA Leiden
        The Netherlands
    


    Deep learning and multimedia retrieval

    Deep Learning Classification demo and publications

    Deep Learning Caption demo and publications


    Publications (technical reports, preliminary work)

    Content-based tag recommendation algorithms for unstructured data

    Improving SIFT accuracy with use of perspective transforms

    Improving the LSDh-tree for fast approximate nearest neighbor search

    Information-Synthesis Network for Facial Landmarks Estimation

    Preliminary Evaluation of CNN Classification by Objective Testers

    Sub-Image Search Engine

    Image Similarity Using Color Histograms

    Rating Inference


    PhD at Leiden University

    PhD Ceremony

    PhD Protocol During Defense

    PhD at Leiden Guidelines

    PhD Training and Compulsory Courses

    ASCI Research School

    CSC and Leiden

    PhD Defense Rules and Guidelines - Cache


    Bus

    Bus - Corpus

    Bus/Train Info

    Bus - Niels Bohrweg

    Bus - Universiteitsterrein

    Bus - De Kempenaerstraat

    Bus - Leidsebuurt

    Bus - Van Assendelftstraat

    Bus - Endegeest


    Links - University

    LIACS Thesis Repository (with students and supervisors index)

    Leiden Teaching Tools

    Committees - Masters

    Lucris/Converis

    Remote Campus (Citrix)

    ServicePlein

    Course Edit

    Leiden Account Info

    SNAPP

    Brightspace

    MyTimeTable

    Leiden Account Services

    Zoom for Leiden

    Edit Courses

    DS Lab details


    Links - General/Random

    MS Edge New Tab Extension

    Media Bias Check

    BuienRadar Pollen

    RIVM Vaccination Overview

    EU Corona Certificate

    Booster Shot

    Corona Vaccinations by Age

    Covid Vaccinations

    Google.com - A good WWW search engine for general information and guides

    Google Search Operators

    Bing Search Operators (e.g. inbody:)

    Thesis Check - Turnitin

    IBM Watson Speech to Text

    Bus Corpus

    Bus Niels Bohrweg

    Dutch Basisverzekering

    Doctor Boender

    DBLP - Michael S. Lew

    DBLP - International Journal of Multimedia Information Retrieval

    MarBo Young on Writing

    Log out Amazon.com

    Log out Amazon.co.uk

    Archive.org - Way Back Machine (historical WWW)

    SAP Leiden

    LIM

    Waist to Height - Penn Health

    Sunblock Ingredients

    Random links

    Lily

    Campaign Promises - BBC

    Campaign Promises - USA Today

    Campaign Promises - Github

    Six Party Talks and Signed Declaration 2005

    GOPDND

    Covid live

    Dilbert

    Perldoc.com - Perl Documentation

    Multimedia Conferences 2011

    AMSR links

    Infinity Web Server

    ACM International Conference on Multimedia Retrieval

    ACM International Conference on Multimedia Retrieval, Glasgow, 2014 Web Archive

    ACM TOMCCAP MIR Survey (vol. 2, issue 1, pp. 1-19, 2006)