Although current large language models are complex, the most basic
specifications of the underlying language generation problem itself are simple
to state: given a finite set of training samples from an unknown l
GPT-2 struggles to learn synthetic impossible languages, challenging the claim that large language models are equally capable of learning languages that are impossible for humans, highlighting the need for further investigation into different LLM architectures for cognitive and typological studies.