Blockify rewrites enterprise RAG: replace chunking with IdeaBlock, compress by 40x, cut tokens by 3x

Enterprise AI data optimization new tool Blockify was compiled and promoted on May 9 by akshay_pachaar, claiming that in a RAG (Retrieval-Augmented Generation) workflow it can compress an enterprise database by 40 times, reduce query token usage by 3 times, and improve vector search accuracy by 2.3 times. Blockify’s official GitHub notes that the product is introduced by Iternal Technologies; it replaces traditional chunking with a structured knowledge unit called “IdeaBlock,” and keeps the knowledge base concise, coherent, and governable through deduplication and merging.

Core concept: replace traditional chunking with IdeaBlock

Blockify’s technical design:

Traditional approach: cut long documents into fixed-size chunks, embed vectors, and retrieve top-k during search

Blockify approach: convert original content into IdeaBlock—XML-structured knowledge units

Each IdeaBlock contains: built-in questions, trusted answers, tags, entities, and keywords

Similar IdeaBlocks are automatically deduplicated and merged, so the knowledge base will not bloat as content grows

The problem with traditional chunking is that the same information may appear in multiple chunks, creating retrieval redundancy and wasting tokens; IdeaBlock improves information density through deduplication—expressing the same content with smaller storage.

Concrete benefits: compress 40 times, reduce tokens by 3 times, and increase accuracy by 2.3 times

Blockify’s published benefit metrics:

Data compression: enterprise database reduced to about 2.5% of the original size (40x compression), retaining 99% or more of the information

Tokens per query: from about 303 (traditional chunks) down to about 98 (IdeaBlock)—3.09x efficiency

Vector retrieval accuracy: increased by 2.29x

Overall accuracy improvement: about 78x (including the combined effects of deduplication and retrieval improvements)

Cost savings calculation: 100 million queries/year, saving approximately $738,000 in token costs

The 78x overall accuracy improvement is a combined effect—deduplication reduces noise, IdeaBlock structured content is friendlier to vector search, and the number of tokens per answer decreases while also reducing the model’s room for errors.

Integration scope: LlamaIndex, LangChain, Milvus, Cloudflare, and other mainstream frameworks

Blockify’s integrated developer tools and infrastructure:

RAG framework: LlamaIndex, LangChain

Knowledge management: Obsidian

Vector database: Milvus, Elastic, Supabase

Edge computing: Cloudflare

Low-code integration: n8n (via workflow templates)

Blockify’s integration strategy is “not to replace existing RAG frameworks, but to act as a pre-data optimization layer.” Developers can, within existing LlamaIndex or LangChain workflows, replace the original chunking step with Blockify while keeping the rest of the pipeline unchanged.

Specific events to watch next: Blockify’s GitHub star growth and community adoption rate; whether Iternal Technologies will apply for or disclose technical details about the IdeaBlock structure (currently branding “patented ingestion”); and whether mainstream RAG frameworks will build similar deduplication logic in as a default feature.

This article, Blockify reworks enterprise RAG: using IdeaBlock instead of chunking, compressing 40x, reducing tokens by 3x, first appeared on Chain News ABMedia.