Product Overview

The Data Flow Index is a cloud-based technology designed for fast and effective processing and analysis of real time and/or large scale spatiotemporal data.

A world of spatiotemporal data

Existing data architecture wasn't designed to manage billions of data records that are created by moving devices as they change location over time.

Instead of finding workarounds for systems that have become slow and expensive, data teams can now make full use of new and historic data with purpose-designed technology.

How does it work?

Plus sign
Ingest batch and live stream spatiotemporal data flows from moving entities onto a single General System instance
Plus sign
Data is indexed during ingest at millions of rows per second making the data instantly available for queries
Plus sign
The multi-dimension index and innovative architecture enables complex queries and event detection to be run on billions of records in seconds, very cost-effectively
Plus sign
Triggers, alerts and results are flowed downstream in real time to improve operations

More about the platform

Based on a highly optimized database kernel containing multiple technologies and unique architectural features, the platform provides an unparalleled foundation and scalable solution for processing high velocity, high volume spatiotemporal workloads.

State-of-art, thread-per-core architecture, vectorised storage model and user space scheduling of I/O and execution combine with the ability to simultaneously ingest, index and query streaming workloads.

Read The Whitepaper
Scale the indexing and storage.

Run spatiotemporal queries on your data including entity ID, latitude and longitude and date/time.

Magnifying glass on paper.

Ingest and filter on up to 8 custom attributes to quickly find records of interest

Magnifying glass over clock.

Store and retrieve unlimited attributes that your data may have

The platform sits alongside your existing data infrastructure....

Batch data

Streaming data

Real time engine

Index

Query

Event detection

Data processing

Data lakes/ Warehouse

Real time apps

Real time events

Deep dive into the platform's unique architecture

4-dimensional index design enables diverse and unrelated spatiotemporal datasets to be easily aggregated. Run complex spatiotemporal queries such as polygon relationships and geofencing on 10-100 billion records in seconds.

Candelstick chart.

Thread-per-core architecture with I/O scheduler designed for high dimension indexes uses its visibility and control over storage access and cache to reorder, optimise and reduce storage operations.

Traffic in a city at sunset.

Continuous, adaptive, background re-sharding distributes the data workload evenly to mitigate overloads and bottlenecks.

A graph of the world's connections.

A high-density storage architecture creates a non-POSIX file system that coexists with standard Linux environments to bypass traditional scalability and performance limitations. The platform processes petabytes of data with consistent performance across millions of logical files while implementing additional features that conventional file systems do not support.

Check out the speed and scale of the solution with a synthetic dataset of 1.6 million vehicles in London. The data is based on vehicle number plates with historic movements loaded in time order to realistically simulate a real-time feed of 92.4 billion datapoints.

Loading.

Data sources supported

People using mobile phones
Physical moving objects
Plus sign

Personal devices, such as a smartphone or smart watch

Plus sign

Fleet of vehicles

Plus sign

Industrial or consumer robots or drones

Plus sign

Weather systems, earthquakes

Graphic
Non physical objects
Plus sign

High volume or automated financial transactions

Plus sign

IP packets and IP traffic patterns

Plus sign

Cyber threats

cameras on a highway
Static sensors that track moving objects
Plus sign

Series of cameras such as number plate recognition

Satellite dish
Aerial or satellite imagery
Plus sign

Indexing pixels or tiles to retrieve specific features at scale

By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.