TimeAtlas Client

Acts as the interface between in-memory RDE model objects and the serialization layer expected by the Time Atlas ingestion pipeline. Entities are grouped by their concrete class and written to individual JSON files, each wrapped in the standard RDE envelope format used throughout the project:

{
    "name": "<filename stem>",
    "type_in_file": ["<rde_type string>"],
    "creation_time": "<ISO-8601 timestamp>",
    "rde_objects": [ ... ]
}

rdes: Flat list of all RDE instances held in this collection.

__init__(rdes)[source]

Create an RDECollection.

Parameters:: rdes (list[RDE]) – List of RDE instances to include in this collection.

add(rdes)[source]

Append one or more RDE instances to the collection.

Parameters:: rdes (list[RDE] | RDE) – A single RDE instance or a list of RDE instances to add.
Return type:: None

save_rde_to_files(output_dir, overwrite=False, rde_types=None, dataset_slug=None)[source]

Serialize the collection’s RDE entities to individual JSON files grouped by type.

For each RDE class present in the collection, one .json file is written to output_dir. The file name matches the keys in _FILE_MAP (e.g. historical_records.json, observations.json).

Serialization relies on each entity’s own to_dict() method. Shapely geometry objects that appear in the resulting dicts (e.g. from Observation or PointOfInterest) are automatically converted to GeoJSON-compatible dicts during the JSON encoding step, so no manual geometry handling is required before calling this method.

If a file already exists and overwrite is False, the serialized rde_objects list is compared with the file’s current content; the file is only rewritten when the content has actually changed. This avoids spurious modification timestamps that would trigger unnecessary downstream reprocessing.

Parameters:

output_dir (str) – Path to the directory where output files are written. The directory (and any missing parents) is created automatically if it does not yet exist.
overwrite (bool) – When True, rewrite every output file unconditionally, even if the content is unchanged. Defaults to False.
rde_types (list[type] | None) – Optional list of RDE classes to serialize (e.g. [HistoricalRecord, Observation]). When None (the default), all types present in the collection are written.
dataset_slug (str | None) – When provided, dataset-tied output files (dataset.json, historical_records.json, observations.json) will include a related_dataset_slugs header field set to [dataset_slug]. When omitted and the collection contains exactly one dataset slug, that slug is inferred automatically.

Raises:

OSError – If output_dir cannot be created or a file cannot be written.

Return type:

None

save_rde_to_jsonl(output_file, overwrite=False, rde_types=None, dataset_slug=None)[source]

Serialize all selected RDE types into one envelope-per-line JSONL package.

Every non-empty type group becomes one compact JSON object on one physical line. Each line has the same envelope shape as a file produced by save_rde_to_files(), including related_dataset_slugs for dataset-scoped resources. Envelope headers are written before rde_objects so a streaming backend can index the package safely.

Parameters:

output_file (str | PathLike) – Destination filename. A .jsonl suffix is added when necessary.
overwrite (bool) – Replace an existing package when True. Defaults to False.
rde_types (Iterable[type[RDE]] | None) – Optional iterable of concrete RDE classes to include.
dataset_slug (str | None) – Optional explicit related dataset slug. When omitted, the only dataset slug in the collection is inferred.

Returns:

The path of the JSONL package.

Raises:

FileExistsError – If the destination exists and overwrite is False.

Return type:

Path

classmethod read_rde_from_files(input_dir, rde_types=None)[source]

Deserialize RDE entities from JSON files produced by save_rde_to_files().

Scans input_dir for .json files, reads each envelope, and reconstructs the appropriate RDE class instances using the rde_type label stored in the type_in_file envelope field. The class is resolved once per file via RDE_TYPE_TO_STATIC_CLASS_DEF, then each object in rde_objects is passed to the matching class’s constructor_from_json_obj classmethod.

Files whose type_in_file value is absent or unrecognised are silently skipped, so partially-populated directories are handled gracefully.

Parameters:

input_dir (str) – Path to the directory containing the serialized .json files.
rde_types (Iterable[type[RDE]] | None) – Optional iterable of concrete RDE classes to deserialize. Files containing other entity types are skipped before their objects are constructed, which is useful when combining several production directories that contain overlapping or very large files.

Returns:

A new RDECollection populated with all successfully deserialized RDE instances.

Raises:

FileNotFoundError – If input_dir does not exist.
json.JSONDecodeError – If a file contains malformed JSON.

Return type:

RDECollection

aggregate_observations_into_points_of_interest(coordinate_precision=5)[source]

Aggregate collection observations into coordinate-based Points of Interest.

Every Observation whose part_of_point_of_interest value is not explicitly False participates in the aggregation. Coordinates are rounded to coordinate_precision decimal places and observations at the same rounded longitude/latitude are assigned the same deterministic UUID. The UUID algorithm and namespace match the legacy merge_obs.py data production utility.

The operation mutates the collection: observation references are replaced by the generated UUID strings and the corresponding PointOfInterest entities are added to rdes. Existing PoIs with a generated UUID are reused so their height information is preserved. PoIs referenced by participating observations under an obsolete UUID are removed, while unrelated PoIs are left untouched. Observations explicitly marked False or lacking a geometry receive a None PoI reference.

Parameters:: coordinate_precision (int) – Number of decimal places used to aggregate coordinates. Five decimal places is the legacy default and gives sub-metre grouping precision.
Returns:: The generated or reused Points of Interest, ordered by rounded longitude and latitude.
Raises:: ValueError – If coordinate_precision is negative or a participating observation has a non-point geometry.
Return type:: list[PointOfInterest]

produce_area_from_current_extent(dataset=None, *, padding=1e-09)[source]

Create and attach an ad-hoc Area covering the collection’s spatial extent.

The extent is the rectangular envelope of the union of every non-empty observation geometry and standalone Geometry RDE. Degenerate point or line extents are padded minimally so the resulting Area always has a polygon geometry suitable for spatial indexing.

When dataset is omitted, the collection must contain exactly one Dataset. The new Area is added to the collection and its UUID is added to dataset.has_areas. UUID and slug generation are deterministic, making repeated calls idempotent.

Parameters:

dataset (Dataset | None)
padding (float)

Return type:

Area

validate_data(mode='strict', *, allow_null_has_geometries=False, allow_unresolved_poi=False)[source]

Validate the internal consistency of all RDE entities in the collection.

First ensures that every Dataset has an area reference for backend spatial indexing. A dataset without one receives a generated, serialized Area covering the current collection extent. Referenced areas absent from the collection emit a warning because they must already exist in the target backend.

Then performs three categories of consistency checks:

Global UUID uniqueness — every entity in the collection must have a distinct UUID.
Array-field UUID uniqueness — within each entity, array fields that hold references to other RDEs must not contain duplicate UUIDs. The affected fields are:
No stale references — whenever an entity references another entity by UUID, that target entity must also be present in the collection. The following reference fields are checked:
- HistoricalRecord.dataset → Dataset
- HistoricalRecord.has_observations → Observation
- Observation.historical_record → HistoricalRecord
- Observation.part_of_point_of_interest → PointOfInterest
- Observation.has_geometries → Geometry
- Layer.map → Map
- Map.layers → Layer
The following fields are intentionally exempt from stale-reference checking because they routinely point to entities outside the collection:
- Geometry.part_of_layer
- Dataset.has_areas
- Map.areas

mode='strict' keeps the default validation rules. mode='raw' is intended for legacy/raw producers and enables both allow_null_has_geometries and allow_unresolved_poi.

Sets _valid_data to True when all checks pass. The method may mutate the collection by adding an extent-derived Area. Any failure raises a ValueError listing every detected problem.

Returns:

True when the collection passes all checks.

Raises:

ValueError – If one or more validation checks fail. The exception message lists every individual problem found.

Parameters:

mode (str)
allow_null_has_geometries (bool)
allow_unresolved_poi (bool)

Return type:

bool

Usage Examples

Basic Usage

from timeatlas import TimeAtlas

# Initialize the client
client = TimeAtlas(api_url='https://your-timeatlas-instance.com/v1')

# Fetch a single RDE object
entity = client.get_single_rde_object('historical-records', 'uuid-here')

# Save entity cache to file
client.save_entity_cache_to_file('my_cache.pkl')

Working with Cached Entities

The TimeAtlas client maintains an internal cache of fetched entities for improved performance. The cache is automatically saved to disk and loaded on subsequent client initializations.

# Cache is automatically loaded from default file on initialization
client = TimeAtlas(api_url='https://api.example.com/v1')

# Save cache to custom location
client.save_entity_cache_to_file('custom_cache.pkl')

Serializing and Validating RDE Collections

RDECollection groups entities for validation and file-based interchange.

from timeatlas import RDECollection

collection = RDECollection([dataset, historical_record, observation])
collection.validate_data()
collection.save_rde_to_files("export", overwrite=True)

restored = RDECollection.read_rde_from_files("export")