Sui gRPC streaming powers low-latency checkpoint indexing and resilient data pipelines

Developers on Sui can now tap into sui grpc streaming to build faster, more reliable indexing pipelines for real-time blockchain data.\n\nHybrid streaming model transforms Sui data access\n\nThe Sui blockchain has introduced gRPC streaming as a primary data source for its indexing infrastructure, enabling real-time checkpoint ingestion with minimal latency. Moreover, this design targets applications that must react as soon as data is finalized.\n\nThe platform combines streaming capabilities with traditional polling methods to maintain data accuracy and system resilience. This hybrid model allows immediate access to finalized checkpoints while preserving backward compatibility with existing custom indexers already deployed on Sui.\n\nThe Custom Indexing Framework underpins this streaming-first architecture without requiring changes to checkpoint processing logic. However, teams can still rely on their existing pipelines while layering in streaming where it brings tangible benefits.\n\nStreaming removes polling delays for checkpoint ingestion\n\nThe new gRPC streaming capability fundamentally changes how indexers receive blockchain data on Sui. Full nodes now push checkpoint data directly to indexers as soon as finalization occurs, rather than waiting for scheduled fetches.\n\nThis push-based model eliminates repeated polling cycles that previously introduced delays between checkpoint creation and downstream processing. As a result, latency-sensitive tools can react closer to real time without tuning complex polling intervals.\n\nAccording to the documentation, the system delivers “real-time checkpoints as soon as they’re finalized” with “faster data, resilient pipelines, less infra work on Sui”. That said, operators can still configure safety nets to protect against connection issues and service disruptions.\n\nThe streaming mechanism operates through a straightforward configuration step where developers add a streaming-url argument that points to a full node endpoint. The indexer then receives checkpoints as event streams instead of fetching them at predetermined intervals.\n\nThis event-driven model is particularly valuable for monitoring systems, real-time analytics platforms, and other latency-sensitive applications. Moreover, it simplifies infrastructure by reducing the need for aggressive polling strategies and related operational tuning.\n\nMandatory polling fallbacks safeguard historical data\n\nSui pairs streaming with mandatory polling-based fallback sources to handle inherent limitations of long-lived connections. A streaming link only delivers data starting from the moment it is established, so historical checkpoints still require additional mechanisms.\n\nThe General-Purpose Indexer showcases this hybrid design in production. It uses streaming as its primary ingestion path while maintaining polling sources as safety mechanisms for historical data and recovery scenarios.\n\nThis configuration keeps indexed data current while enabling clean restarts and seamless recovery from failures. However, if a connection drops, the system can resume from the last known checkpoint using polling, then return to streaming once the link stabilizes.\n\nIn practice, this hybrid pattern functions similarly to a sui checkpoint streaming fallback strategy. Developers obtain the low-latency benefits of pushed updates without compromising completeness or reliability.\n\nFramework design enables incremental streaming adoption\n\nThe Custom Indexing Framework separates checkpoint processing from data ingestion. Indexers consume and transform checkpoints through a unified interface, without coupling logic to specific sources such as gRPC streams or HTTP polling.\n\nThis abstraction allows teams to evolve ingestion strategies as requirements change, without rewriting core processing components. Moreover, it keeps codebases simpler by concentrating data-handling logic in a single layer.\n\nThe documentation notes that with gRPC streaming, “there is no need to poll, no guesswork around timing, and no artificial delay introduced by fetch intervals”. That said, operators can still choose polling where workloads do not require ultra-low latency.\n\nDevelopers can enable sui grpc streaming gradually, based on individual workload characteristics. Applications that prioritize data freshness and real-time responsiveness gain the most from immediate streaming adoption.\n\nBy contrast, systems focused on batch analytics, offline processing, or simpler workflows can continue using polling-only configurations. The framework supports both approaches under the same processing model, simplifying multi-application environments.\n\nConfiguration keeps logic stable while sources evolve\n\nExisting custom indexers built on the official framework require minimal changes to take advantage of streaming. Adding gRPC capability involves including a streaming-url parameter alongside the existing remote-store-url configuration value.\n\nThe checkpoint processing logic remains unchanged throughout this transition. Moreover, the framework automatically manages source switching during operation so that indexers keep a consistent view of the network state.\n\nThis design helps prevent common failure modes where systems either lose data or lag significantly behind the chain. The framework coordinates interactions between streaming and polling, maintaining continuity across restarts and network interruptions.\n\nOverall, Sui’s hybrid streaming and polling architecture offers real-time checkpoints, resilient pipelines, and a clear migration path for indexers seeking low-latency ingestion without sacrificing reliability.