First Steps for Data Analysis / Visualization #9

New Issue

phryk · 2023-01-23T10:36:38Z

phryk commented

2023-01-23 10:36:38 +00:00

We should re-evaluate the data modeling from poobrains.
With Postgres/PostGIS as a more specialized storage solution,
we can probably save individual datapoints as rows with a single
index (probably a Foreign Key pointing to a table where each
row represents a dataset).

This would also make combinations, comparisons etc. of datasets
easier but makes it more complicated to handle columns of different
types within datasets.

Potential approach:

Datasets are based on "templates" that specify their structure.
Each dataset template corresponds to an automatically created
table where the values of all datapoints of all datasets using
this template are stored.

Has other complications, tho:

How do we handle changes to a template without losing existing data?
Either needs working database introspection (preferable) or a robust notation for the structure

Other questions:

Do we still have to maintain the separation between maps and plots?
Window functions (i.e. range limiting) for visualization?
How do we best do automatic LODing?
- PostGIS has various simplify functions
- for numeric values maybe one of Postgres' array aggregation functions?

We should re-evaluate the data modeling from poobrains. With Postgres/PostGIS as a more specialized storage solution, we can probably save individual datapoints as rows with a single index (probably a Foreign Key pointing to a table where each row represents a dataset). This would also make combinations, comparisons etc. of datasets easier but makes it more complicated to handle columns of different types within datasets. Potential approach: > Datasets are based on "templates" that specify their structure. > Each dataset template corresponds to an automatically created > table where the values of all datapoints of all datasets using > this template are stored. Has other complications, tho: * How do we handle changes to a template without losing existing data? * Either needs working database introspection (preferable) or a robust notation for the structure Other questions: * Do we still have to maintain the separation between maps and plots? * Window functions (i.e. range limiting) for visualization? * How do we best do automatic LODing? * PostGIS has various `simplify` functions * for numeric values maybe one of Postgres' array aggregation functions?

phryk added this to the Data visualization and analysis base package milestone 2023-11-07 01:56:45 +00:00

Sign in to join this conversation.

No Label

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: phryk/ooze#9