Tech Overview

At the core of the General System platform is a multi-attribute index and innovative storage features architected for large scale multi-dimensional analysis

A world of spatiotemporal data

Access data via the dfipy Python SDK or REST API in any language. Batch and streaming spatiotemporal datasets (ID, latitude, longitude, altitude, date/time + 8 optional filter fields) are loaded into a single table which negates the need for data partitioning and multiple table lookups. Queries to identify or count records within a specified area are executed in seconds instead of hours, enabling fast analysis of locations, routes, dwells and co-location.

The GS platform excels in real-time, contextual querying across datasets. Imagine projects identifying anomalies or comparing trends that previously took hours or days, running programmatically every minute!

How does it work?

Plus sign
Batch and live stream data is indexed on ingest at millions of rows per second for immediate querying in real time. The high-density storage architecture enables scaling to hundreds of billions of records on a single instance.
Plus sign
The multi-attribute index on ID, latitude, longitude and time ordering processes queries exceptionally fast, even at massive scale. No need to partition data, no index bloat, no need to recluster and tune.
Plus sign
Continuous, adaptive, background resharding distributes the data workload evenly to mitigate hotspotting and overloads which saves laborious partitioning and tuning.
Plus sign
Thread-per-core architecture with I/O scheduler uses its visibility and control over storage access and cache to reorder, optimise and reduce storage operations to save hardware and maintenance.

More about the platform

Based on a highly optimized database kernel containing multiple technologies and unique architectural features, the platform provides an unparalleled foundation and scalable solution for processing high velocity, high volume spatiotemporal workloads.

State-of-art, thread-per-core architecture, vectorised storage model and user space scheduling of I/O and execution combine with the ability to simultaneously ingest, index and query streaming workloads.

Read The Whitepaper
Scale the indexing and storage.

Based on proven, reliable AWS architecture

Magnifying glass on paper.

Extremely high levels of security with robust user and data access permissions

Magnifying glass over clock.

Visualization via standard industry tools

The platform complements your existing data infrastructure....

Batch data

Streaming data

Real time engine



Event detection

Data processing

Data lakes/ Warehouse

Real time apps

Real time events

Processing Data in General System vs PostGIS

Technology that is purposed-designed for managing spatiotemporal data can provide significant benefits.

Comparison Posts

PostGIS - based on ingestion of 17 billion AdTech data records on a db.r6g.16xlarge server with 5 metadata fields.
General System based on ingestion of 101 billion AdTech data records on
an i3en.12xlarge server with 6 metadata fields. Costs include ETL and ingestion costs.

Deep dive into the platform's unique architecture

4-dimensional index design enables diverse and unrelated spatiotemporal datasets to be easily aggregated. Run complex spatiotemporal queries such as polygon relationships and geofencing on 10-100 billion records in seconds.

Candelstick chart.

Thread-per-core architecture with I/O scheduler designed for high dimension indexes uses its visibility and control over storage access and cache to reorder, optimise and reduce storage operations.

Traffic in a city at sunset.

Continuous, adaptive, background re-sharding distributes the data workload evenly to mitigate overloads and bottlenecks.

A graph of the world's connections.

A high-density storage architecture creates a non-POSIX file system that coexists with standard Linux environments to bypass traditional scalability and performance limitations. The platform processes petabytes of data with consistent performance across millions of logical files while implementing additional features that conventional file systems do not support.

Check out the speed and scale of the solution with a synthetic dataset of 1.6 million vehicles in London. The data is based on vehicle number plates with historic movements loaded in time order to realistically simulate a real-time feed of 92.4 billion datapoints.


Supported data types

Augment streaming or batch spatiotemporal data from multiple sources

People using mobile phones
Physical moving objects
Plus sign


Plus sign


Plus sign

Robots and drones

Plus sign

GPS trackers

Non physical objects
Plus sign

High volume or automated financial transactions

Plus sign

IP packets and IP traffic patterns

Plus sign

Cyber threats

cameras on a highway
Sensors that track moving objects
Plus sign

Static sensors that track moving objects

Plus sign

Series of cameras such as number plate recognition

Satellite dish
Aerial or satellite imagery
Plus sign

Indexing pixels or tiles to retrieve specific features at scale

By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.