about image

Yong-Lu Li

Ph.D. Student

Email: yonglu_li[at]sjtu[dot]edu[dot]cn

[Google Scholar] [Github] [LinkedIn]

[ResearchGate] [dblp] [Semantic Scholar]

Machine Vision and Intelligence Group

Shanghai Jiao Tong University

About

I’m a forth-year PhD student in the Department of Computer Science at Shanghai Jiao Tong University (SJTU). Currently I am a member of Machine Vision and Intelligence Group (MVIG) under the supervision of Prof. Cewu Lu. Prior to that, I worked and studied in Institute of Automation, Chinese Academy of Sciences (CASIA) under the supervision of Prof. Yiping Yang and A/Prof. Yinghao Cai (3D Reconstruction). My primary research interests are Computer Vision and Robotics. Currently, I focus on developing a knowledge-driven vision system (HAKE) which learns to effectively perceive human activities, reason human behavior logics and interact with objects and environments.



News



2020.09: Our work HOI Analysis will appear at NeurIPS 2020. Code and arXiv paper are comming soon.

2020.07: Fortunate to recieve WAIC YunFan Award and be among the 2nd A-Class Project.

2020.06: Larger HAKE-Large (>120K images with activity and part state labels) is released!

2020.04: The code&data and paper of "Detailed 2D-3D Joint Representation for Human-Object Interaction (CVPR2020)" are released!

2020.04: The code, data and paper of our image-based HAKE, "PaStaNet: Toward Human Activity Knowledge Engine (CVPR2020)", are released!

2020.04: The code and paper of our CVPR 2020 paper Symmetry and Group in Attribute-Object Compositions are available now!

2020.02: Three papers Image-based HAKE: PaSta-Net, 2D-3D Joint HOI Learning, Symmetry-based Attribute-Object Learning are accepted in CVPR 2020! Papers and corresponding resources (code, data) will be released soon.

2019.08: Code for our ICCV2019 paper InstaBoost is available now!

2019.07: Our paper InstaBoost is accepted in ICCV 2019.

2019.06: The Part I of our HAKE : HAKE-HICO which contains the image-level part-state annotations is released!

2019.06: Code for our CVPR2019 paper on Human-Object Interaction is available now!

2019.04: Our dataset Instance-60k & 3D Object Models in ECCV2018 paper SRDA is available!

2019.04: Our project HAKE: Human Activity Knowledge Engine begins trial operation!

2019.02: Our paper on Interactiveness is accepted in CVPR 2019.

2018.11: We released our new paper on Interactiveness.

2018.07: Our paper on GAN & Annotation Generation is accepted in ECCV 2018.

2018.05: Presentation (Kaibot Team) in TIDY UP MY ROOM CHALLENGE | ICRA 2018.

2018.02: Our paper on Object Part States is accepted in CVPR 2018.

2017.07: I joined the MVIG lab.




Publications





HAKE: Human Activity Knowledge Engine

Yong-Lu Li, Liang Xu, Xinpeng Liu, Xijie Huang, Yue Xu, Mingyang Chen, Ze Ma, Shiyi Wang, Hao-Shu Fang, Cewu Lu.

Preprint  [arXiv] [PDF] [Project] [Code]

DecAug: Augmenting HOI Detection via Decomposition

Yichen Xie, Hao-Shu Fang, Dian Shao, Yong-Lu Li, Cewu Lu.

Preprint  [arXiv] [PDF]

HOI Analysis: Integrating and Decomposing Human-Object Interaction

Yong-Lu Li*, Xinpeng Liu*, Xiaoqian Wu, Yizhuo Li, Cewu Lu (*=equal contribution).

NeurIPS 2020  Coming soon: [arXiv] [PDF] [HAKE-Action-Torch] [Code]

PaStaNet: Toward Human Activity Knowledge Engine

Yong-Lu Li, Liang Xu, Xinpeng Liu, Xijie Huang, Yue Xu, Shiyi Wang, Hao-Shu Fang, Ze Ma, Mingyang Chen, Cewu Lu.

CVPR 2020  [arXiv] [PDF] [Code] [Data] [Video] [Slides]

Oral Talk, Compositionality in Computer Vision in CVPR 2020.

Detailed 2D-3D Joint Representation for Human-Object Interaction

Yong-Lu Li, Xinpeng Liu, Han Lu, Shiyi Wang, Junqi Liu, Jiefeng Li, Cewu Lu.

CVPR 2020  [arXiv] [PDF] [Code] [Benchmark: Ambiguous-HOI] [Video] [Slides]

Symmetry and Group in Attribute-Object Compositions

Yong-Lu Li, Yue Xu, Xiaohan Mao, Cewu Lu.

CVPR 2020  [arXiv] [PDF] [Code] [Video] [Slides]

InstaBoost: Boosting Instance Segmentation Via Probability Map Guided Copy-Pasting

Hao-Shu Fang*, Jianhua Sun*, Runzhong Wang*, Minghao Gou, Yong-Lu Li, Cewu Lu.

ICCV 2019  [arXiv] [PDF] [Code]

Transferable Interactiveness Knowledge for Human-Object Interaction Detection

Yong-Lu Li, Siyuan Zhou, Xijie Huang, Liang Xu, Ze Ma, Hao-Shu Fang, Yan-Feng Wang, Cewu Lu.

CVPR 2019  [arXiv] [PDF] [Code]

SRDA: Generating Instance Segmentation Annotation via Scanning, Reasoning and Domain Adaptation

Wenqiang Xu*, Yong-Lu Li*, Cewu Lu. (*=equal contribution).

ECCV 2018  [arXiv] [PDF] [Code] [Dataset](Instance-60k & 3D Object Models)

Beyond Holistic Object Recognition: Enriching Image Understanding with Part States

Cewu Lu, Hao Su, Yong-Lu Li, Yongyi Lu, Li Yi, Chi-Keung Tang, Leonidas J. Guibas.

CVPR 2018  [PDF]

Optimization of Radial Distortion Self-Calibration for Structure from Motion from Uncalibrated UAV Images

Yong-Lu Li, Yinghao Cai, Dayong Wen, Yiping Yang.

ICPR 2016  [PDF]



Projects





HAKE: Human Activity Knowledge Engine

Human Activity Knowledge Engine (HAKE) aims at promoting human activity understanding. As a large-scale knowledge base, HAKE is built upon existing activity datasets and provides human body part level atomic action labels (Part States). Given a human box, Activity2Vec will convert it into a fixed-size vector combining visual and linguistic features for diverse downstream tasks, i.e. image/video action recognition/detection, captioning, VQA, visual reasoning, image retrieval, etc. Conventional instance-based methods enhanced with HAKE outperform SOTA approaches on several large-scale activity benchmarks (HICO, HICO-DET, V-COCO, AVA, etc). Now we are enriching HAKE to make it become a general research platform of knowledge extraction and causal inference. Come and join us!

[Work in Progress]  [Website]



Public Services





Reviewer

Conference: CVPR'20/21, NeurIPS'20, AAAI'21, ACCV'20, WACV'21.

Journal: Neurocomputing, JVCI.



Talks





2020.8.23: Knowledge Driven Human Activity Understanding

The 3rd International conference on Image, Video Processing and Artificial Intelligence, IVPAI 2020.

2020.7.12: Human Activity Knowledge Engine

Student Forum on Frontiers of AI, SFFAI. [Slides]

2020.6.16: PaStaNet: Toward Human Activity Knowledge Engine

Compositionality in Computer Vision in CVPR 2020 Virtual. [Video] [Slides]



Honors





China National Scholarship, Sep. 2020.

WAIC YunFan Award, Rising Star, July. 2020 (World Artificial Intelligence Conference, Shanghai).

The 2nd A-Class Project , July. 2020.

Annual Scholarship, SJTU, Nov. 2019.



Personal Interests





Chinese History Reading

Twenty-Four Histories: 5/24 (･ิω･ิ)

Early Four Historiographies: Records of the Grand Historian, Book of Han, Records of the Three Kingdoms, Book of Later Han;
The Book of Jin;

Travel around China

34 Provinces: 20/34 (･ิω･ิ)