HE Xin (贺鑫), a.k.a. MARSGGBO

Driven by a mission to democratize deep learning, my research is dedicated to advancing the accessibility and efficiency of large-scale deep learning models, particularly Large Language Models (LLMs). My goal is to bridge the theoretical aspects of machine learning with practical system designs to create scalable, robust, and trustworthy AI systems that are widely accessible. My interested research directions include:

1.Model-Centric AI:
- Architecture Dearch: Neural Architecture Search (e.g., multi-objective NAS, Training-free NAS, resource-aware NAS), Sparse Model (e.g., Mixture-of-Experts)
- Hyper-parameter optimization (HPO): Grid/Random Search, Evolutionary Algorithm, Differentiable Optimization
- Model Compression: Pruning, Quantization, Knowledge Distillation
2.Data-Centric AI:
- Automatic Data Augmentation (ADA), Data Generation, Dataset compression,
- RAG, LLM alignment
3.HPC AI:
- Memory efficiency: Offloading, KV-cache
- LLM training acceleration: Distributed Parallellism (data parallel, tensor parallel, pipeline parallel)
- LLM inference optimization: Batch Schedule, Dynamic Inference Paths

Education & Work

🇸🇬 National university of Singapore (NUS), Singapore (2023-now)
- Research Fellow, School of Computing
- Principle Investigator: Prof. You Yang
🇭🇰 Hong Kong Baptist University (HKBU), Hong Kong, China (2018 - 2023)
- Ph.D, Department of Computer Science
- Supervisor: Prof. Chu Xiaowen
🇨🇳 Huazhong University of Science & Technology (HUST), Wuhan, China (2014 - 2018)
- Bachelor, School of Electronic Information and Communications, Honor of Outstanding Student

Interested in collaboration? Contact me.

Highlights

1.AutoML Survey (1400+ citation): AutoML: A Survey of State-of-ther-art
2.AutoML Framework: Hyperbox
3.AutoML Applications:
- AAAI2021 COVID3DNet: The first NAS application for COVID-19 3D CT scans
- MICCAI2022 EMARS: Evolutionary algorithm-based NAS for COVID-19 3D CT scans
- ECCV2022 EAGAN: Two-stage NAS for GANs
- AAAI2023 NAS-LID: NAS with Local Intrinsic Dimension

news

Feb 23, 2024	I am honored to serve as a reviewer for ECCV’24.
Sep 03, 2023	Our paper “MedPipe: End-to-End Joint Search of Data Augmentation Policy and Neural Architecture for 3D Medical Image Classification” is accepted to IEEE MedAI2023!
Aug 09, 2023	I am honored to serve as a reviewer for AAAI’24.
Nov 19, 2022	Our paper “NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension” is accepted to AAAI2023!
Nov 02, 2022	I am honored to serve as a reviewer for CVPR’23.

latest posts

Jun 26, 2024	Deepspeed ZeRO系列算法原理+通信开销详解 - marsggbo
Jun 09, 2024	NSCC集群使用笔记 - marsggbo
May 06, 2024	Huggingface Transformers实现张量并行的小坑 set/get_output_embeddings - marsggbo

selected publications

AAAI

NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension

Xin He , Jiangchao Yao , Yuxin Wang , and 5 more authors

AAAI, 2023

Bib PDF Code

@article{NASLID,
  title = {NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension},
  author = {He, Xin and Yao, Jiangchao and Wang, Yuxin and Tang, Zhenheng and Ka Chun, Cheung and Simon, See and Han, Bo and Chu, Xiaowen},
  booktitle = {AAAI},
  year = {2023},
  journal = {AAAI},
  corr = {true},
  zhihu = {https://zhuanlan.zhihu.com/p/591027254},
  special = {CCF-A},
  gamma = {https://gamma.app/public/0t1h94j75bccc0q},
  dimensions = {true},
  google_scholar_id = {Tyk-4Ss8FVUC}
}

KBS

AutoML: A survey of the state-of-the-art

Xin He , Kaiyong Zhao , and Xiaowen Chu

Knowledge-Based Systems, 2021

Bib PDF Website

@article{he2021automl,
  title = {AutoML: A survey of the state-of-the-art},
  author = {He, Xin and Zhao, Kaiyong and Chu, Xiaowen},
  journal = {Knowledge-Based Systems},
  volume = {212},
  pages = {106622},
  year = {2021},
  publisher = {Elsevier},
  special = {CCF-C, 1300+ citations by Oct 2023.},
  zhihu = {https://zhuanlan.zhihu.com/p/158162306},
  corr = {true},
  dimensions = {true},
  google_scholar_id = {zYLM7Y9cAGgC}
}

AAAI

Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans

Xin He , Shihao Wang , Xiaowen Chu , and 6 more authors

In AAAI , 2021

Bib PDF

@inproceedings{He2021AutomatedMD,
  title = {Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans},
  author = {He, Xin and Wang, Shihao and Chu, Xiaowen and Shi, Shaohuai and Tang, J. and Liu, Xin and Yan, Chenggang Clarence and Zhang, Jiyong and Ding, Guiguang},
  booktitle = {AAAI},
  year = {2021},
  corr = {true},
  special = {CCF-A},
  dimensions = {true},
  google_scholar_id = {2osOgNQ5qMEC},
}

MICCAI

Evolutionary Multi-objective Architecture Search Framework: Application to COVID-19 3D CT Classification

Xin He , Guohao Ying , Jiyong Zhang , and 1 more author

In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) , 2022

Bib PDF Code

@inproceedings{emars,
  title = {Evolutionary Multi-objective Architecture Search Framework: Application to COVID-19 3D CT Classification},
  author = {He, Xin and Ying, Guohao and Zhang, Jiyong and Chu, Xiaowen},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)},
  pages = {560--570},
  year = {2022},
  corr = {true},
  special = {CCF-B},
  zhihu = {https://zhuanlan.zhihu.com/p/542168478},
  dimensions = {true},
  google_scholar_id = {UeHWp8X0CEIC},
  special_old = {top premier international conferences for medical image analysis}
}