Blockify Technical Documentation
Patented data ingestion, distillation, and governance pipeline designed to optimize unstructured enterprise content for use with Retrieval-Augmented Generation (RAG) and other AI/LLM applications.
System Overview
Blockify transforms text documents into small, semantically complete "IdeaBlocks." The system uses patented algorithms to ingest, distill, and govern enterprise content, making it optimized for RAG and other AI/LLM applications.
LLM Models
Blockify uses two primary LLM models designed for different stages of the pipeline:
Blockify Ingest
Converts raw text chunks into structured XML IdeaBlocks with comprehensive metadata.
Blockify Distill
Merges semantically similar IdeaBlocks while removing redundancy and preserving accuracy.
Available Versions
Technical Specifications
| Component | Parameter | Value |
|---|---|---|
| Blockify Ingest | Input Size | 1,000-4,000 characters (2,000 recommended) |
| Data Fidelity | ~99% lossless for numerical data, facts, and key information | |
| Output | Structured XML IdeaBlocks with metadata | |
| Blockify Distill | Input Size | 2-15 IdeaBlocks per request |
| Function | Remove duplicates, separate distinct concepts | |
| Data Fidelity | ~99% lossless for numerical data, facts, and key information |
System Requirements
Compute Options
- CPU: Xeon Series 4, 5, or 6
- GPU: Intel Gaudi 2/3
- GPU: NVIDIA
- GPU: AMD
Software Dependencies
- MLops/LLM runtime supporting LLAMA
- Any embeddings model (OpenAI, Mistral, Jina, AWS)
- Any vector database (Milvus, Pinecone, Azure, AWS)
- Any parsing/chunking system (Unstructured.io, LangChain)
Chunking Guidelines
Default Chunk Size
1,000-4,000 characters (2,000 recommended for optimal processing)
Technical Documentation
4,000 characters recommended for comprehensive technical content
Meeting Transcripts
4,000 characters to capture full context and speaker transitions
Chunk Overlap
10% overlap recommended to maintain context between chunks
Split Strategy
Split at logical boundaries (paragraphs, sections) for best results
API Configuration
Recommended API settings for optimal performance:
output_tokens: 8000+
temperature: 0.5
format: OpenAPI standard
Licensing Model
User-based licensing options for flexible deployment:
Note: All data processed through Blockify must remain for internal use only unless explicitly licensed otherwise.
Deployment Process
Download and Unzip
Download the Blockify LLM package and extract the contents
Upload and Convert
Upload and convert to the required format for your MLops platform
Deploy on MLops
Deploy the model on your preferred MLops platform
Test Inference
Verify deployment with test inference calls
Commercial Licensing & Support
For commercial licenses, deployment assistance, or technical support
Contact Support Team