Zhaoyang Xia
Ph.D. student @ Rutgers | zx149@rutgers.edu

I am a Ph.D. student in the Department of Computer Science at Rutgers University, advised by Distinguished Professor Dimitris Metaxas. Piror to this, I finished my Master Program in Computer science (Data Science track) at Rutgers. I studied mathamtics and data science when I was an undergraduate student in Fudan University.

My current research interests lie primarily in generative models, such as diffusion models and Large Language Models. I am also interested in representation learning, video/image understanding, such as action detection, object segmentation or action recognition. My previous projects are about utilizing generative models and image animation models for applications in video anonymization. I also have experience in using multi-modal large language models for applications such as visual storytelling and ideation.

  |     |  


Education

Sept. 2021 - Present
Rutgers Univeristy
Ph.D. Student in Computer Science (GPA: 4.0/4.0)
Sept. 2019 - May. 2021
Rutgers Univeristy
Master Student in Computer Science (Data Science) (GPA: 3.91/4.0)
Sept. 2015 - Jun. 2019
Fudan University
B.S. in Information and Computing Science (Big Data & Technology)

Experience

May. 2024 - Aug. 2024
Adobe Inc.
Research Scientist/Engineer Intern. Mentors: Somdeb Sarkhel, Mehrab Tanjim, Stefano Petrangeli, Ishita Dasgupta, Saayan Mitra
  • Designed novel generative AI for visual storytelling methods (visual story ideation) using multi-modal large language models. Paper in submission
May. 2023 - Dec. 2023
Adobe Inc.
Research Scientist/Engineer Intern. Mentors: Somdeb Sarkhel, Stefano Petrangeli, Uttaran Bhattacharya, Nikos Vlassis, Saayan Mitra
  • Designed algorithms for improving diffusion models with human preference data
Sept. 2020 - Present
Rutgers University. Piscataway, NJ, USA
Research Assistant. Advisor: Prof. Dimitris N. Metaxas
  • DiffSLVA: Harnessing Diffusion Models for Sign Language Video Anonymization
    [LREC'24 Workshop]
  • American Sign Language Video Anonymization to Support Online Participation of Deaf and Hard of Hearing Users
    [ASSET]
  • Region Proposal Rectification Towards Robust Instance Segmentation of Biological Images.[MICCAI'21]
Oct. 2018 - Aug. 2019
Fudan University. Shanghai, China
Research Assistant. Advisor: Prof. Deqing Yang
  • Explainable Recommendation System for Movies
  • The Knowledge Graph Integration
Oct. 2018 - Feb. 2019
MISUMI (China) precision machinery trading co., LTD, Shanghai, China
Data Science Intern.
  • Predict the loss of customers based on customer behavior data.
  • Chat context analysis: Find the hot topics from the chat history by NLP technology.

Publications

DiffSLVA: Harnessing Diffusion Models for Sign Language Video Anonymization
[Paper] [Demos] [Bibtex]
Sign Language Video Anonymization
Zhaoyang Xia, Yuxiao Chen, Qilong Zhangli, Matt Huenerfauth, Carol Neidle and Dimitri Metaxas
Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2022)
[Paper] [Demos] [Bibtex]
American sign language video anonymization to support online participation of deaf and hard of hearing users
Sooyeon Lee, Abraham Glasser, Becca Dingman, Xia, Zhaoyang, Dimitris Metaxas, Carol Neidle, and Matt Huenerfauth.
ASSET
[Paper] [Demos] [Bibtex]
Hierarchically self-supervised transformer for human skeleton representation learning
Yuxiao Chen, Long Zhao, Jianbo Yuan, Yu Tian, Xia, Zhaoyang, Shijie Geng, Ligong Han, and Dimitris N. Metaxas.
ECCV 2022
[Paper] [Bibtex]
TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers
Di Liu, Yunhe Gao, Qilong Zhangli, Ligong Han, Xiaoxiao He, Xia, Zhaoyang, Song Wen, Qi Chang, Zhennan Yan, Mu Zhou, et al.
MICCAI 2022
[Paper] [Bibtex]
Region Proposal Rectification Towards Robust Instance Segmentation of Biological Images
Qilong Zhangli, Jingru Yi, Di Liu, Xiaoxiao He, Zhaoyang Xia, Haiming Tang, He Wang, Mu Zhou, Dimitris Metaxas
MICCAI 2022
[Paper] [Bibtex]

Templete stolen from Jon Barron and Georgia Gkioxari