
Product
A world of spatiotemporal data
- Indexing billions of small records quickly
- The need to search across multiple columns and attributes at scale
- Unpredictable distribution of data and query loads causing storage hotspots
- Poor storage cache efficiency
Challenges working with live location data:
As data volumes grow and new data sources are added, the complexity, time and cost of data aggregation, ingest, storage and query, make processing impractical.
About the platform
Read The WhitepaperCode snippets
Snippet 1 - Add a new observation
In this case London Congestion Zone with 700 vertices
new_observation = [
{
‘coordinate’: [112.325, 81.578, 23], # [Long, Lat, Altitude ],‘time’: ‘2022-09-01T17:32:28.250Z’,‘id’: ‘e9ed6717-51d3-476f-a7a1-a8cd0237f080’,‘payload’: ‘{“custom_field”, “custom_value” }’,‘instance’ : ‘my_dataset’
}
]
r = requests.post(
"https://api.dataflowindex.io/insert",
json=new_observation,
headers={"X-API-TOKEN": f"Bearer {api_token}")
Snippet 2 - Find all observations in a polygon
In this case London Congestion Zone with 700 vertices
query_polygon = "london_congestion_charge_zone"
query_parameters = {
‘startTime’: ‘2022-01-01T08:00:00.000Z’,
‘endTime’: ‘2022-01-01T09:30:00.000Z’,
‘instance’ : ‘my_dataset’
}
r = requests.get(
f"https://api.dataflowindex.io/polygon/{query_polygon}/history",
params=my_params,
headers={"X-API-TOKEN": f"Bearer {api_token}",
stream = True)
Snippet 3 - Find all unique entities in a polygon over a specific timeframe.
In this case London Congestion Zone with 700 vertices
query_polygon = "london_congestion_charge_zone"
query_parameters = {
‘instance’ : ‘my_dataset’
}
r = requests.get(
f"https://api.dataflowindex.io/polygon/{query_polygon}/entities",
params=my_params,
headers={"X-API-TOKEN": f"Bearer {api_token}",
stream = True)
# returns list of entities IDs
Deep dive into the platform's unique architecture

Check out the speed and scale of the solution with a synthetic dataset of 1.6 million vehicles in London. The data is based on vehicle number plates with historic movements loaded in time order to realistically simulate a real-time feed of 92.4 billion datapoints.
Batch data
Streaming data
Real time engine
Index
Query
Event detection


Data processing
Data lakes/ Warehouse
Real time apps
Real time events
Data sources supported

Physical moving objects
Personal devices, such as a smartphone or smart watch
Fleet of vehicles
Industrial or consumer robots or drones
Weather systems, earthquakes

Non physical objects
High volume or automated financial transactions
IP packets and IP traffic patterns
Cyber threats

Static sensors that track moving objects
Series of cameras such as number plate recognition

Aerial or satellite imagery
Indexing pixels or tiles to retrieve specific features at scale
Spatiotemporal Glossary
An entity is any object that sends data such as an individual with mobile phone, vehicle wih GPS tracker, shipping container or parcel with tracking number, or IoT sensor. The platform assumes that the user wants to model unique and groups of entities as they move (or track movement) through space over time. Each entity has a unique identifier such as mobile phone number, vehicle registration number plate or machine serial number.
These are self-contained repositories for your data containing a table of columns for ID, Latitude, Longtitude, Altitude, a timestamp and optional payload of additional information, such as measurement taken.
An entity such as a GPS tracker will send its position every few seconds, sometimes with telemetry data. These records are called recordings or observations. The platform can ingest millions of observations a second, subject to the available network bandwidth and local disk I/O throughput.
If an entity ID has multiple observations, the set of data points is referred to as the entity's 'history'.
Create a polygon around a geographical area of interest, such as a building, shopping mall, bridge, road or border, to retrieve all entities (and history of those entities) in the polygon during the timeframe specified. A bounding box query does the same but is limited to a simple 4-sided shape with 2 longtitude and 2 latitude co-ordinates.