Skip to content

Commit 87a02eb

Browse files
chrissetochrisseto
committed
Prettify tables
1 parent 8ef1f8e commit 87a02eb

1 file changed

Lines changed: 57 additions & 57 deletions

File tree

‎whitepapers/Tables.md‎

Lines changed: 57 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
### {ModelName}
66
{Description}
77

8-
| Column | Type | Indexed | Nullable | FK | Default | Description |
9-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
10-
| | | ✓ | ✓ | ✓ | | |
8+
| Column | Type | Indexed | Nullable | FK | Default | Description |
9+
| :----- | :--: | :-----: | :------: | :-: | :-----: | :---------- |
10+
| | | ✓ | ✓ | ✓ | | |
1111

1212
#### Other indices
1313
* `{column_name}`, `{column_name}`, ... [(unique)]
@@ -18,98 +18,98 @@
1818
### SourceUniqueIdentifier (SUID)
1919
Identifier for a specific document from a specific source.
2020

21-
| Column | Type | Indexed | Nullable | FK | Default | Description |
22-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
23-
| `identifier` | text | | | | | Identifier given to the document by the source |
24-
| `ingest_config_id` | int | | | ✓ | | IngestConfig used to ingest the document |
21+
| Column | Type | Indexed | Nullable | FK | Default | Description |
22+
| :----------------- | :--: | :-----: | :------: | :-: | :-----: | :--------------------------------------------- |
23+
| `identifier` | text | | | | | Identifier given to the document by the source |
24+
| `ingest_config_id` | int | | | ✓ | | IngestConfig used to ingest the document |
2525

2626
#### Other indices
2727
* `source_doc_id`, `ingest_config_id` (unique)
2828

2929
### RawData
3030
Raw data, exactly as it was given to SHARE.
3131

32-
| Column | Type | Indexed | Nullable | FK | Default | Description |
33-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
34-
| `suid_id` | int | | | ✓ | | SUID for this datum |
35-
| `data` | text | | | | | The raw data itself (typically JSON or XML string) |
36-
| `sha256` | text | unique | | | | SHA-256 hash of `data` |
37-
| `harvest_logs` | m2m | | | | | List of HarvestLogs for harvester runs that found this exact datum |
32+
| Column | Type | Indexed | Nullable | FK | Default | Description |
33+
| :------------- | :--: | :-----: | :------: | :-: | :-----: | :----------------------------------------------------------------- |
34+
| `suid_id` | int | | | ✓ | | SUID for this datum |
35+
| `data` | text | | | | | The raw data itself (typically JSON or XML string) |
36+
| `sha256` | text | unique | | | | SHA-256 hash of `data` |
37+
| `harvest_logs` | m2m | | | | | List of HarvestLogs for harvester runs that found this exact datum |
3838

3939
## Ingest Configuration
4040

4141
### IngestConfig
4242
Describes one way to harvest metadata from a Source, and how to transform the result.
4343

44-
| Column | Type | Indexed | Nullable | FK | Default | Description |
45-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
46-
| `source_id` | int | | | ✓ | | Source to harvest from |
47-
| `base_url` | text | | | | | URL of the API or endpoint where the metadata is available |
48-
| `earliest_date` | date | | ✓ | | | Earliest date with available data |
49-
| `rate_limit_allowance` | int | | | | 5 | Number of requests allowed every `rate_limit_period` seconds |
50-
| `rate_limit_period` | int | | | | 1 | Number of seconds for every `rate_limit_allowance` requests |
51-
| `harvester_id` | int | | | ✓ | | Harvester to use |
52-
| `harvester_kwargs` | jsonb | | ✓ | | | JSON object passed to the harvester as kwargs |
53-
| `transformer_id` | int | | | ✓ | | Transformer to use |
54-
| `transformer_kwargs` | jsonb | | ✓ | | | JSON object passed to the transformer as kwargs, along with the harvested raw data |
55-
| `disabled` | bool | | | | False | True if this ingest config should not be run automatically |
44+
| Column | Type | Indexed | Nullable | FK | Default | Description |
45+
| :--------------------- | :---: | :-----: | :------: | :-: | :-----: | :--------------------------------------------------------------------------------- |
46+
| `source_id` | int | | | ✓ | | Source to harvest from |
47+
| `base_url` | text | | | | | URL of the API or endpoint where the metadata is available |
48+
| `earliest_date` | date | | ✓ | | | Earliest date with available data |
49+
| `rate_limit_allowance` | int | | | | 5 | Number of requests allowed every `rate_limit_period` seconds |
50+
| `rate_limit_period` | int | | | | 1 | Number of seconds for every `rate_limit_allowance` requests |
51+
| `harvester_id` | int | | | ✓ | | Harvester to use |
52+
| `harvester_kwargs` | jsonb | | ✓ | | | JSON object passed to the harvester as kwargs |
53+
| `transformer_id` | int | | | ✓ | | Transformer to use |
54+
| `transformer_kwargs` | jsonb | | ✓ | | | JSON object passed to the transformer as kwargs, along with the harvested raw data |
55+
| `disabled` | bool | | | | False | True if this ingest config should not be run automatically |
5656

5757
### Source
5858
A Source is a place metadata comes from.
5959

60-
| Column | Type | Indexed | Nullable | FK | Default | Description |
61-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
62-
| `name` | text | unique | | | | Short name |
63-
| `long_title` | text | unique | | | | Full, human-friendly name |
64-
| `home_page` | text | | ✓ | | | URL |
65-
| `icon` | image | | ✓ | | | Recognizable icon for the source |
66-
| `user_id` | int | | | ✓ | | User with permission to submit data as this source (TODO: replace with django permissions stuff) |
60+
| Column | Type | Indexed | Nullable | FK | Default | Description |
61+
| :----------- | :---: | :-----: | :------: | :-: | :-----: | :----------------------------------------------------------------------------------------------- |
62+
| `name` | text | unique | | | | Short name |
63+
| `long_title` | text | unique | | | | Full, human-friendly name |
64+
| `home_page` | text | | ✓ | | | URL |
65+
| `icon` | image | | ✓ | | | Recognizable icon for the source |
66+
| `user_id` | int | | | ✓ | | User with permission to submit data as this source (TODO: replace with django permissions stuff) |
6767

6868
### Harvester
6969
Each row corresponds to a Harvester implementation in python. (TODO: describe those somewhere)
7070

71-
| Column | Type | Indexed | Nullable | FK | Default | Description |
72-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
73-
| `key` | text | unique | | | | Key that can be used to get the corresponding Harvester subclass |
74-
| `date_created` | datetime | | | | now | |
75-
| `date_modified` | datetime | | | | now (on update) | |
71+
| Column | Type | Indexed | Nullable | FK | Default | Description |
72+
| :-------------- | :------: | :-----: | :------: | :-: | :-------------: | :--------------------------------------------------------------- |
73+
| `key` | text | unique | | | | Key that can be used to get the corresponding Harvester subclass |
74+
| `date_created` | datetime | | | | now | |
75+
| `date_modified` | datetime | | | | now (on update) | |
7676

7777
### Transformer
7878
Each row corresponds to a Transformer implementation in python. (TODO: describe those somewhere)
7979

80-
| Column | Type | Indexed | Nullable | FK | Default | Description |
81-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
82-
| `key` | text | unique | | | | Key that can be used to get the corresponding Transformer subclass |
83-
| `date_created` | datetime | | | | now | |
84-
| `date_modified` | datetime | | | | now (on update) | |
80+
| Column | Type | Indexed | Nullable | FK | Default | Description |
81+
| :-------------- | :------: | :-----: | :------: | :-: | :-------------: | :----------------------------------------------------------------- |
82+
| `key` | text | unique | | | | Key that can be used to get the corresponding Transformer subclass |
83+
| `date_created` | datetime | | | | now | |
84+
| `date_modified` | datetime | | | | now (on update) | |
8585

8686
## Logs
8787

8888
### HarvestLog
8989
Log entries to track the status of a specific harvester run.
9090

91-
| Column | Type | Indexed | Nullable | FK | Default | Description |
92-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
93-
| `ingest_config_id` | int | | | ✓ | | IngestConfig for this harvester run |
94-
| `harvester_version` | text | | | | | Semantic version of the harvester, with each segment padded to 3 digits (e.g. '1.2.10' => '001.002.010')
95-
| `start_date` | datetime | | | | | Beginning of the date range to harvest |
96-
| `end_date` | datetime | | | | | End of the date range to harvest |
97-
| `started` | datetime | | | | | Time `status` was set to STARTED |
98-
| `status` | text | | | | INITIAL | Status of the harvester run, one of {INITIAL, STARTED, SPLIT, SUCCEEDED, FAILED} |
91+
| Column | Type | Indexed | Nullable | FK | Default | Description |
92+
| :------------------ | :------: | :-----: | :------: | :-: | :-----: | :------------------------------------------------------------------------------------------------------- |
93+
| `ingest_config_id` | int | | | ✓ | | IngestConfig for this harvester run |
94+
| `harvester_version` | text | | | | | Semantic version of the harvester, with each segment padded to 3 digits (e.g. '1.2.10' => '001.002.010') |
95+
| `start_date` | datetime | | | | | Beginning of the date range to harvest |
96+
| `end_date` | datetime | | | | | End of the date range to harvest |
97+
| `started` | datetime | | | | | Time `status` was set to STARTED |
98+
| `status` | text | | | | INITIAL | Status of the harvester run, one of {INITIAL, STARTED, SPLIT, SUCCEEDED, FAILED} |
9999

100100
#### Other indices
101101
* `ingest_config_id`, `harvester_version`, `start_date`, `end_date` (unique)
102102

103103
### TransformLog
104104
Log entries to track the status of a transform task
105105

106-
| Column | Type | Indexed | Nullable | FK | Default | Description |
107-
|:-------|:----:|:-------:|:--------:|:--:|:-------:|:------------|
108-
| `raw_id` | int | | | ✓ | | RawData to be transformed |
109-
| `ingest_config_id` | int | | | ✓ | | IngestConfig used |
110-
| `transformer_version` | text | | | | | Semantic version of the transformer, with each segment padded to 3 digits (e.g. '1.2.10' => '001.002.010')
111-
| `started` | datetime | | | | | Time `status` was set to STARTED |
112-
| `status` | text | | | | INITIAL | Status of the transform task, one of {INITIAL, STARTED, RESCHEDULED, SUCCEEDED, FAILED} |
106+
| Column | Type | Indexed | Nullable | FK | Default | Description |
107+
| :-------------------- | :------: | :-----: | :------: | :-: | :-----: | :--------------------------------------------------------------------------------------------------------- |
108+
| `raw_id` | int | | | ✓ | | RawData to be transformed |
109+
| `ingest_config_id` | int | | | ✓ | | IngestConfig used |
110+
| `transformer_version` | text | | | | | Semantic version of the transformer, with each segment padded to 3 digits (e.g. '1.2.10' => '001.002.010') |
111+
| `started` | datetime | | | | | Time `status` was set to STARTED |
112+
| `status` | text | | | | INITIAL | Status of the transform task, one of {INITIAL, STARTED, RESCHEDULED, SUCCEEDED, FAILED} |
113113

114114
#### Other indices
115115
* `raw_id`, `transformer_version` (unique)

0 commit comments

Comments
 (0)