Decoding Market Sentiment for 100 Million Users: How We Build a Multi-Model AI Engine with 1-Second Response

In the world of cryptocurrency, a misinterpreted news story can lead to millions of dollars in misjudgments. Our legacy sentiment analysis system—a hybrid architecture combining open-source models with self-hosted LLMs—has become overwhelmed when faced with real-time news streams in 25 languages worldwide. A typical failure scenario occurs when events like “Ethereum Merge” generate completely opposite interpretations within different language communities. Our system either experiences delayed responses or produces conflicting sentiment labels. This has prompted us to rethink our core approach: how to provide fast and accurate market insights to a global user base. The answer ultimately points to a carefully designed “multi-model consensus” architecture.

Source: InterSystems

Architecture Evolution: From a Single Model to an Expert Committee

We initially fell into the trap of searching for a “universal model.” Practical experience proved that no single LLM could meet production-level requirements simultaneously in processing speed, multilingual accuracy, and cryptocurrency domain knowledge. Claude 3 Haiku responds quickly but has limited understanding of slang in Chinese communities; our fine-tuned Mistral model excels at parsing whitepapers but faces bottlenecks in long-text throughput. More critically, self-hosting these models imposes infrastructure burdens—GPU resource competition during peak traffic and ongoing operational complexity—draining our team. These pain points drove us toward the core concept of model federation: enabling specialized models to do their jobs and integrating collective intelligence through an intelligent arbitration mechanism.

Dual-Path Asynchronous Pipeline Design

The new system’s core is a dual-path asynchronous pipeline running on AWS, designed with the philosophy of strictly controlling P99 latency within seconds while ensuring redundancy.

News texts first enter two processing channels in parallel. The first is a high-speed channel that directly invokes Claude 3 Haiku on Amazon Bedrock for initial sentiment judgment and key entity extraction, typically within 300 milliseconds. The second is a deep analysis channel that sends the text to a fine-tuned Mistral 7B model on Amazon SageMaker for domain context enhancement—such as distinguishing whether “gas fee surge” is due to network congestion or popular NFT minting—this process takes about 600 milliseconds.

The real innovation lies in the lightweight arbitration layer. It compares outputs from both paths in real-time. When results are highly consistent, it prioritizes the high-speed channel to ensure rapid response; when discrepancies occur, it makes a decision within 20 milliseconds based on preset domain rules and confidence scores. This mechanism ensures that most requests receive reliable insights that are both fast and deep within one second.

The Hidden Battlefield of Data Pipelines

Building the models themselves is only the surface of engineering challenges; the real complexity lies within the data pipeline. Data streams from global news sources and social media are noisy, filled with multilingual content, emojis, and internet slang. We built a multi-layer filtering system—combining language-specific regular expressions with real-time detection models based on FastText—to ensure input text cleanliness. The stability of this preprocessing directly affects the confidence of subsequent analysis.

A greater challenge is establishing an evaluation system. We rely not only on manual annotations by multilingual expert teams but also incorporate market reactions as dynamic validation indicators: correlating sentiment outputs with short-term asset price fluctuations to continuously optimize evaluation standards. This shifts the system from pursuing static annotation accuracy to tracking the effectiveness of dynamic market perception.

Infrastructure Cost Philosophy

Migrating to the Bedrock API has fundamentally changed our operational model. The most significant benefit is the complete elimination of infrastructure burdens and near-infinite elastic scalability—when sudden news causes traffic to surge by 300%, the system can respond smoothly without manual intervention. Although the cost structure is based on token usage, by intelligently caching high-frequency narrative templates and continuously optimizing prompt engineering, overall expenses are reduced by approximately 35% compared to idle GPU clusters in self-hosted setups. This shift frees engineering resources, allowing focus on arbitration logic and pipeline optimization—core areas of innovation.

Conclusion and Future Directions

The key insight from this architecture evolution is that for high-performance production systems, “a single authoritative model” is often less effective than “a committee of specialized experts.” By organically combining the responsiveness of general LLMs with the deep semantic understanding of domain-specific models, we have finally built a sentiment perception system capable of withstanding real-time global market challenges.

Looking ahead, we are driving the system from “sentiment analysis” toward an “narrative tracking” intelligence. The new challenge is enabling AI not only to judge sentiment polarity but also to identify and continuously track the formation, diffusion, and decay of emerging narratives such as “real-world asset tokenization.” This requires architectures with stronger memory mechanisms and causal reasoning capabilities, leading us toward the frontier of next-generation intelligent financial infrastructure.