Differential syntactic and semantic encoding in LLMs

arXiv:2601.04765v1 Announce Type: new Abstract: We study how syntactic and semantic information is encoded in inner layer representations of Large Language Models (LLMs), focusing on the very large DeepSeek-V3. We find that, by averaging hidden-representation vectors of sentences sharing syntactic …