CV

Education

• 2020.09 - 2024.06 ~ Beijing University of Posts and Telecommunications (BUPT)
  • School of information and communication engineering, Communication Engineering, Class for talent
• 2024.08 - 2028.06 ~ The Chinese Univeristy of Hong Kong
  • Department of Computer Science and Engineering

Internships

• 2021.06-2022.12 Intelligent System Research Group in BUPT
  • Interests: NLP and Knowledge graph (supervised by Chunhong Zhang and Xinning Zhu)
  • Duty: I have studied the Pytorch, Linux, and NLP models. We proposed an automated knowledge extraction for data extraction, data cleaning, and graph fusion framework over 100 knowledge graphs. The paper got best paper award nomination in ICALT2023.
• 2022.05-2023.03 National Engineering Research Center of Mobile Network Security in BUPT
  • Interest: Multimodal and Semantic communication (supervised by Prof. Guoshun Nan and Xiaofeng Tao)
  • Duty: I do research in multimodal information fusion and semantic communication systems. We proposed channel-layer black-box attack method targeting semantic communication systems and a unified framework for semantic protection. Our 3 papers have been accepted by VTC2022fall, ICC2023 and IEEE Communication Magazine. In 2023, we proposed a multimodal semantic communication system that utilizes text information assisting in image transmission and reconstruction.

• 2023.03-2023.05 Graph Data Mining and Machine Learning Lab (GAMMA Lab) in BUPT

  • Interest: Data Mining and GNN (supervised by Prof. Chuan Shi)
  • Duty: I study GNN and graph data mining.

• 2023.04-2023.07 The National Laboratory of Pattern Recognition (NLPR) at the Institute of Automation of the Chinese Academy of Sciences (CASIA).

  • Interest: Diffusion model and AI4Science.
  • Duty: I do research in diffusion model, AIG and AI4Science.

• 2023.07-2024.04 GAP Lab, Chinese University of Hong Kong (Shenzhen)

  • Interest: AIGC and 3d generation. (supervised by Prof.Xiaoguang Han)
  • Duty: I do research in image and 3d generation. We work on the topic ’Sketch to 3D’, using text prompts and sketch to controllable generate high-quality 3D objects. I finish my bachelor thesis in GAP lab.
• 2023.08-Now Beijing Shuyuan Research Center
  • Interest: multi-modal, 3d scene understanding and LLM.
  • Duty: I do research in understanding and perception of 3D scenes, and inject visual and multi-modal information into LLM to create a multi-modal 3D LLM. The 3D LLM can perceive and understand the scene and the relationships between objects within the scene, making downstream tasks more effective. Our work : 3DMIT (ICME-3DMM 2024), SeeingMore (ECCV2024).
• 2023.04-Now King Abdullah University of Science and Technology (KAUST)
  • Interest: LLM continual learning (supervised by Mohamed Elhoseiny)
  • Duty: I do research in LLM continual learning, transfer learning as a visiting student. .
• 2023.05-Now CUHK & Nanjing EDA Research Center
  • Interest: Code-LLM, tokenizer, RTL LLM alignment. (supervised by Qiang Xu)
  • Duty: I do research in RTL(verilog) represent learning and understanding by LLM, alignment of HLS and RTL, Large Circuit Model.

Publications

• Zeju Li, Chao Zhang, Xiaoyan Wang, Ruilong Ren, Yifan Xu, Ruifei Ma, Xiangde Liu. 3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding.(ICME-3DMM 2024), [pdf] [code]
• Zeju Li, Xinghan Liu, Guoshun Nan* , Jinfei Zhou, Xinchen Lyu, Qimei Cui, Xiaofeng Tao. Boosting Physical Layer Black-Box Attacks with Semantic Adversaries in Semantic Communications. Proceedings of the International Communication Conference. (ICC 2023) [pdf]
• Zeju Li, Linya Cheng, Chunhong Zhang, Xinning Zhu, Hui ZhaoMulti-source Education Knowledge Graph Construction and Fusion for College Curricula.Proceedings of the IEEE International Conference on Advanced Learning Technologies. (ICALT 2023) [pdf]
• Zeju Li, Jinfei Zhou, Guoshun Nan* , Zhichun Li, Qimei Cui, Xiaofeng Tao. SemBAT: Physical Layer Black-box Adversarial Attacks for Deep Learning-based Semantic Communication Systems. 2022 IEEE 96th Vehicular Technology Conference. (VTC 2022-fall) [pdf]
• Xinghan Liu, Guoshun Nan*, Qimei Cui, Zeju Li, Peiyuan Liu, Zebin Xing, Hanqing Mu, Yuanwei Liu, Xiaofeng Tao. SemProtector: A Unified Framework for Semantic Protection in Deep Learning-based Semantic Communication Systems. (IEEE Communciation Magazine)(SCI Q1)[pdf]
• Zhiyuan Wu, Sheng Sun, Min Liu, Quyang Pan, Junbo Zhang, Yuwei Wang*, Zeju Li, and Qingxiang Liu. Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation. (ACM Transactions on Intelligent Systems and Technology)(CAA Rank A、SCI Q1)[pdf]

Awards

• Best Paper Award Nomination by The IEEE Technical Community on Learning Technology(2023) [certificate]

• Singapore International Pre-Graduate Award(SIPGA) 2023
• BUPT-SICE Excellent Student Creative Foundation 2023
• Outstanding Graduate of BUPT 2024
• Candidates for Special Scholarship in BUPT 2024

• Full Postgraduate Studentship in CUHK 2024

• People’s Daily Scholarship 2023

• China National Endeavor Scholarship (2021,2022,2023)

• The 1st prize of the National Software and Information Technology Professionals Competition of python program design in the Blue Bridge Cup[certificate]

• The 1st prize of the 9th China International “Internet plus” Undergraduate Innovation and Entrepreneurship Competition

• The 1st prize of Innovation and Entrepreneurship Practice Achievement Exhibition and Exchange Conference for College Students[certificate]

• The 1st prize of “Foreign Language Teaching Society · Word Talent Cup” National College Students’ English Vocabulary Ability Competition[certificate]

• The 2nd prize of the 33rd Beijing University Student Mathematics Competition [certificate]

• The 2nd prize of National Education Cup National Undergraduate Mathematics Competition [certificate]

• The 2nd prize of the National College Student New Generation Information and Communication Technology Competition of Datang Cup [certificate]

• The 2nd prize of Chinese Chemistry Olympiad[certificate]

• The 3rd prize of National Undergraduate Mathematics Competition [certificate]

• The 3rd prize of physics competition of Beijing University of Posts and Telecommunications in 2021 [certificate]

• The 3rd prize of Young Goose Program Innovation and Entrepreneurship Competition Final [certificate]

• Excellence Award of Internet plus Innovation and Entrepreneurship Competition in Beijing [certificate]

Projects

• The Smart Wardrobe based on deep learning

  • Abstract: The intelligent wardrobe has a built-in clothing protection function by an intelligent temperature and humidity control module in the wardrobe; Moreover, the app can query the real-time weather and temperature conditions to recommend clothes suitable for users to wear today through internet.
  • Duty: Mainly responsible for algorithm
• The learning assistant “byr” based on deep learning
  • Abstract: A software with the main functions of exercise recommendation system, personnel activity question-answering system, course knowedge retrieval system, mathematical formula recognition and recording the results in the back-end and database.
  • Duty: Mainly responsible for algorithm and the backend by django
• Research of 3D multimodal-LLM by injecting 3D scenes and objects
  • Abstract: This project proposes to combine the reasoning and dialogue capabilities of 3D visual perception models and Large Language Models (LLM) to achieve a universal multimodal-LLM for 3D scenes. And we sovle the problem of unbalance of visual and language training. We align and fuse the features of the text image pair with the segmented objects in the corresponding 3D scene at a fine-grained level. After that, the features of the relative position relationship matrix between objects were obtained using the coordinates of each object in the scene. Finally, the language model is injected with multimodal knowledge (features) as learnable visual tokens to be added to the language tokens in the middle of each Transformer layer of the LLM to fine-tune and train the 3D multimodal large model of this project.
• Semantic communication robustness based on adversarial learning and deep learning
  • Abstract: We propose a practical physical layer black-box attack method for deep learning-based semantic communication systems, where the adversaries are imperceptible to humans. Our work has been accepted by ICC2023.
  • Duty: Mainly responsible for algorithm design, conduct experiments and paper writing
• Research of discipline knowledge graphs in the field of Electronic Information
  • Abstract: We propose an automated knowledge extraction, visual knowledge graph construction in Neo4j and graph fusion framework for Electronic Information. Moreover, we develop other functions such as exercise recommendation system, personnel activity question-answering system, course knowedge retrieval system and mathematical formula recognition.
  • Duty: Mainly responsible for knowledge graphs construction, data cleaning, graphs fusion and paper writing.