Learning When to Think While Listening in Large Audio-Language Models
ICLR 2027Under reviewInternational Conference on Learning Representations
My name is Zhiyuan Song, but you can also call me Allen. I am currently an M.S. student in Computer and Information Science (AI Track) at the University of Pennsylvania. I am fortunate to work as a Research Assistant advised by Prof. Jiatao Gu, focusing on multimodal foundation models and large language model reasoning.
My primary research interests lie in building intelligent, real-time interactive systems. My current focus spans LLM post-training, Omni & multi-model systems, real-time interaction models, agent harnesses, continual learning, and long-horizon tasks. I am also passionate about robust AI infrastructure and optimizing distributed training pipelines.
I completed my undergraduate studies at the University of Virginia, where I double majored in Computer Science and Applied Statistics and graduated with High Distinction, and was fortunate to conduct research advised by Prof. Madhav Marathe, Prof. S.S. Ravi, and Prof. N. Rich Nguyen.
I welcome any entrepreneurial ideas and am always open to finding co-founders for exciting ventures. Feel free to reach out — I'd love to chat!
Education
University of Pennsylvania
Master of Science, Computer and Information Science (AI Track).
University of Virginia
Bachelor of Arts with High Distinction, Computer Science & Applied Statistics.
Honors: Dean's List; UVA Grad App Fund Scholarship; NSF Fund.
Experience
Millennium Management
AI Engineer Intern — Cloud & AI/ML Infrastructure
Entropinia
Founder — my own brand and studio; built Legal Manus, a RAG legal assistant
News
Completed my summer internship at Millennium Management and got an extension — continuing as an AI Engineer Intern.
Started as an AI Engineer Intern at Millennium Management, working on cloud & AI/ML infrastructure.
New preprint: "Learning When to Think While Listening in Large Audio-Language Models" is on arXiv, under review at ICLR 2027.
I started my M.S. in CIS at UPenn and joined Prof. Jiatao Gu's group as a Research Assistant.
Led the team as Tech Lead & MLE at AdventureX Hackathon, building a full-duplex real-time audio AI assistant.
Graduated from University of Virginia with High Distinction.
Paper "SL4HOI", with Prof. N. Rich Nguyen, accepted at KDD 2024 in Barcelona.
Founded Entropinia — it began as a clothing brand and grew into my own studio, where I built AI products such as Legal Manus.
Joined the UVA Biocomplexity Institute as a Research Assistant under Prof. Madhav Marathe and Prof. S.S. Ravi.
Selected Publications
SL4HOI: Supervised Learning for Predicting Health Opportunity Index across States
KDD 2024Knowledge Discovery and Data Mining, Barcelona
