Claude Generates 3,000 Lines of Custom Code Instead of Importing pywikibot
Avoid letting Claude generate large custom libraries; instead, prompt it to import existing packages.
Prompt Claude to import existing libraries and verify output before running.
Summary
Claude wrote roughly 3,000 lines of Python to reimplement pywikibot, mwparserfromhell, and Wikipedia’s RETF ruleset instead of importing the existing PyPI packages. The custom code included a wikitext stripper, a typo dictionary, RETF rules, and ten edit runners, all of which were unnecessary because upstream libraries already provide these features.
After debugging, the author manually migrated the code down to 1,259 lines by creating shims over mwparserfromhell and pywikibot and fetching RETF rules at runtime. Claude retained an 18‑entry typo dictionary even though all entries existed in RETF, illustrating the model’s tendency to preserve redundant work. The article explains that benchmarks penalize library imports, leading the model to generate custom code when it cannot reach the internet or pip. This pattern repeats in other contexts, such as Claude writing custom SVG instead of using a charting library. The post highlights the importance of prompting the model to use existing libraries and verifying its output before execution. It also discusses how the model’s “sunk‑cost defense” keeps large code blocks in context, making it harder to prune them.
Key changes
- Claude generated ~3,000 lines of custom Python reimplementing pywikibot, mwparserfromhell, and RETF ruleset
- Reimplemented wikitext stripper, typo dictionary, RETF rules, and ten edit runners, all of which exist in upstream libraries
- Failed to import existing PyPI packages, leading to redundant code
- After manual migration, code reduced to 1,259 lines using shims over mwparserfromhell and pywikibot
- Retained an 18‑entry typo dictionary despite it being present in RETF
- Model’s benchmarks penalize library imports, causing it to write custom code when it cannot access the internet or pip
- Demonstrates a “sunk‑cost defense” where large code blocks remain in context, making pruning difficult
- Highlights the need to prompt the model to import libraries and verify output before execution