Talkie 13B Vintage Language Model – 1930‑Era Text Generation

Talkie is a 13B language model trained on 260 B tokens of pre‑1931 English text, released as talkie‑1930‑13b‑base (53.1 GB) and a fine‑tuned chat version talkie‑1930‑13b‑it (26.6 GB). Both models are Apache 2.0 licensed and hosted on Hugging Face. The base model was trained on out‑of‑copyright historical texts, while the chat model was fine‑tuned using instruction‑response pairs extracted from pre‑1931 reference works and synthetic prompts. The fine‑tuning process involved Claude Sonnet 4.6 as a judge and a second round of supervised fine‑tuning with Claude Opus 4.6 and talkie.

Research objectives include: - Predicting future events using a model trained on historical data. - Assessing whether the model can invent concepts beyond its knowledge cutoff. - Evaluating the model’s ability to write correct Python programs from a few demonstrations.

The team emphasizes the challenge of avoiding contamination from post‑1931 text and modern LLM assistance. They plan to bootstrap future training with vintage models themselves to reduce anachronistic influence.

Key points: - Base model: 13B, 260 B tokens, 53.1 GB. - Chat model: 13B, 26.6 GB. - Fine‑tuned with Claude Sonnet 4.6 and Claude Opus 4.6. - Research on future prediction, invention, and programming. - Apache 2.0 license. - Training data entirely out of copyright. - Plans to release training corpus or scripts.

The article includes a demo link and a Hacker News discussion.

The project showcases how vintage data can be used to build LLMs.

The article was published in May 2026.

The article includes tags such as ai, generative‑ai, local‑llms, llms, training‑data, ai‑ethics, llm‑release.