66
77Schemas describe the structure of ingested data. They are used in the pipeline to validate the types
88and values of data, and to define a table schema in a data store. We use a repository of JSON
9- Schemas to sort incoming data into [ ` decoded ` and ` error ` datasets] [ bq-datasets ] . We also generate
10- BigQuery table schemas on business days from the JSON Schemas: you can see the current status of
11- this job on the [ ` mozilla-pipeline-schemas ` deploy dashboard] [ mps-deploys ] .
9+ Schemas to sort incoming data into [ ` decoded ` and ` error ` datasets] [ bq-datasets ] .
1210
1311``` mermaid
1412graph TD
@@ -74,9 +72,7 @@ probe-scraper will automatically pick up changes from `metrics.yaml`.
7472
7573Schema deploys happen on business days around UTC+04 when new changes are found in the
7674[ ` generated-schemas ` branch of ` mozilla-pipeline-schemas ` ] [ generated-schemas ] . This means that any
77- changes merged after UTC+04 on Friday will not propagate until Monday UTC+04. See the
78- [ ` mozilla-pipeline-schemas ` deploy] [ mps-deploys ] dashboard for up-to-date information on the most
79- recent deploys.
75+ changes merged after UTC+04 on Friday will not propagate until Monday UTC+04.
8076
8177### What does it mean when a schema deploy is blocked?
8278
@@ -94,9 +90,8 @@ is not registered before collection begins, then it will be sorted into the erro
9490may be affected by blocked schema deploys.
9591
9692[ bq-datasets ] : ../../cookbooks/bigquery/querying.md#projects-with-bigquery-datasets
97- [ mps-deploys ] : https://protosaur.dev/mps-deploys/
9893[ mps ] : https://github.com/mozilla-services/mozilla-pipeline-schemas
99- [ generated-schemas ] : https://github.com/mozilla-services/mozilla-pipeline-schemas/tree/generated-schema
94+ [ generated-schemas ] : https://github.com/mozilla-services/mozilla-pipeline-schemas/tree/generated-schemas
10095[ msg ] : https://github.com/mozilla/mozilla-schema-generator
10196[ probe-scraper ] : https://github.com/mozilla/probe-scraper
10297
0 commit comments