Dear Zevist family, I present to you: Arcane Forum Archiver >>> a recursive crawling and mirroring engine
that walks an entire vBulletin board — every section, every thread, every page, every avatar, every attachment — and binds it into a permanent,
self-contained offline archive. No database to maintain, no server to pay for, no dependency on a site that might simply not be there tomorrow.
This isn't a slapped-together script to "save a couple of pages." It's a full desktop application with a graphical interface, 7 operational modules
(we call them the Seven Rites, in keeping with the rest of the project), built to hold up on boards with 10,000+ posts without falling apart halfway through a scan.
____________________________________________________
⚙️ TECHNICAL ARCHITECTURE — what's under the hood
____________________________________________________
- Heuristic crawl depth control
CSS-selector-driven parsing that tracks threadid, postid, page to
know exactly how deep to go and when to stop. Automatic stripping of
session tokens and redundant parameters — so zero infinite loops, zero
duplicates.
- In-memory deduplication
Hashing sets track every visited and pending endpoint visitedpending),
so nothing is ever fetched or written twice.
- Dynamic media mirroring
Every remote src and href is rewritten to a local path on the fly,
with MIME validation via HTTP header analysis to guarantee the integrity
of downloaded images, avatars, and attachments.
- Asynchronous disk writes
Binary persistence runs off the main thread, so ingesting thousands of
files never freezes the interface.
- Multi-threading
Crawling and downloading run on dedicated daemon threads, keeping the
Tkinter UI responsive even during massive scans.
- Manual garbage collection
Cyclic calls to gc.collect() and BeautifulSoup buffer flushing keep
long runs stable across tens of thousands of posts.
___________________________________________
🔮 THE SEVEN RITES — the program's 7 tabs
___________________________________________
RITE I — Forum Backup
Recursively walks every section, thread, and page of a
vBulletin board, downloading posts, avatars, and attachments.
RITE II — Convert → HTML
Transforms raw archived pages into clean, locally navigable
HTML, reconstructing thread coherence with zero external
dependencies.
RITE III — Backup + Convert
The full pipeline in a single pass: crawl, mirror media, and
render to HTML, unattended.
RITE IV — Restoration Ritual
Recovers structured data (author, timestamp, content,
hierarchy) from previously archived pages and re-exports it
as clean JSON or CSV.
RITE V — Website Mirror
A general-purpose mirroring worker that normalizes URLs and
rewrites every asset reference into a self-contained local
copy of any site — not just vBulletin.
RITE VI — Static Site Fusion
Fuses hundreds of separately archived pages into one unified,
cross-linked static site, with assets embedded (including as
base64) and an auto-generated navigation tree. One single
file, zero dependencies, still openable twenty years from now.
RITE VII — PDF Unifier
Binds archived documents together into a single merged PDF —
the final, printable form of a preserved forum.
____________________________
💻 PLATFORMS & REQUIREMENTS
____________________________
Windows Standalone .exe, x86_64. No Python installation required.
Linux Single-file binary, x86_64. Requires python3-tk at runtime.
Source Pure Python, MIT licensed. Requires Python 3.13 / 3.14 with
tkinter, requests, and bs4 if you want to build it
yourself (buildable with PyInstaller, spec files included in
the repo).
____________________________
📦 WHERE TO FIND EVERYTHING
____________________________
🌐 Official page (direct downloads + live crawl log demo):
💻 GitHub repo (full source, MIT, issue tracker):
📚 A first "live" archive already preserved with this same tool, on
Internet Archive:
____________________________________________________
Here's why I'm saying this with this much emphasis: a forum only lives as long as its host's hosting bill, its domain, and the will of whoever runs it keeps going. With this tool, any one of you — even without ever touching a line of code — can take a complete, permanent copy of this community: threads, images, attachments, structure — and carry it offline, forever. One double-click, even with no internet, even twenty years from now.
____________________________________________________