Skip to content

Commit 6330d48

Browse files
committed
Moved Reports to data types file
1 parent b0af9c1 commit 6330d48

3 files changed

Lines changed: 128 additions & 107 deletions

File tree

data-types.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ This MDS data types page catalogs the objects (fields, types, requirements, desc
1515
- [Stops](#stops)
1616
- [Stop Status](#stop-status)
1717
- [Trips](#trips)
18+
- [Reports](#reports)
1819

1920
## Vehicles
2021

@@ -222,6 +223,59 @@ A Trip is defined by the following structure:
222223

223224
[Top][toc]
224225

226+
## Reports
227+
228+
A Report is defined by the following structure:
229+
230+
| Column Name | Type | Comments |
231+
|----------------------| ----------------------------------------- | ------------------------------------------------ |
232+
| `provider_id` | UUID | A UUID for the Provider, unique within MDS. See MDS provider_id in [provider list](/providers.csv). |
233+
| `start_date` | date | Start date of trip the data row, ISO 8601 date format, i.e. YYYY-MM-DD |
234+
| `duration` | string | Value is always `P1M` for monthly. Based on [ISO 8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations) |
235+
| `special_group_type` | [Special Group Type](#special-group-type) | Type that applies to this row |
236+
| `geography_id` | [Geography](/geography) | ID that applies to this row. Includes all IDs in /geography. When there is no /geography then return `null` for this value and return counts based on the entire operating area. |
237+
| `vehicle_type` | [Vehicle Type](/general-information.md#vehicle-types) | Type that applies to this row |
238+
| `trip_count` | integer | Count of trips taken for this row |
239+
| `rider_count` | integer | Count of unique riders for this row |
240+
241+
[Top][toc]
242+
243+
### Data Notes
244+
245+
Report contents include every combination of special group types, geography IDs, and vehicle types in operation for each month since the provider began operations in the jurisdiction. New files are added monthly in addition to the previous monthly historic files.
246+
247+
Counts are calculated based the agency's local time zone. Trips are counted based on their start time, i.e. if a trip starts in month A but ends in month B, it will be counted only as part of the report for month A. Similarly, trips are counted based on their start geography, i.e. if a trip starts in geography A and ends in geography B, it will appear in the counts for geography A and not for geography B.
248+
249+
All geography IDs included in the city published [Geography](/geography) API endpoint are included in the report results. In lieu of serving an API, this can alternately be a [flat file](/geography#file-format) created by the city and sent to the provider via link. If there is no `/geography` available, then counts are for the entire agency operating area, and `null` is returned for each Geography ID.
250+
251+
[Top][toc]
252+
253+
### Data Redaction
254+
255+
Some combinations of parameters may return a small count of trips, which could increase a privacy risk of re-identification. To correct for that, Reports does not return data below a certain count of results. This data redaction is called k-anonymity, and the threshold is set at a k-value of 10. For more explanation of this methodology, see our [Data Redaction Guidance document](https://github.com/openmobilityfoundation/mobility-data-specification/wiki/MDS-Data-Redaction).
256+
257+
**If the query returns fewer than `10` trips in a count, then that row's count value is returned as "-1".** Note "0" values are also returned as "-1" since the goal is to group both low and no count values for privacy.
258+
259+
This value may be adjusted in future releases and/or may become dynamic to account for specific categories of use cases and users. To improve the specification and to inform future guidance, users are encouraged to share their feedback and questions about k-values on this [discussion thread](https://github.com/openmobilityfoundation/mobility-data-specification/discussions/622).
260+
261+
Using k-anonymity will reduce, but not necessarily eliminate the risk that an individual could be re-identified in a dataset, and this data should still be treated as sensitive. This is just one part of good privacy protection practices, which you can read more about in our [MDS Privacy Guide for Cities](https://github.com/openmobilityfoundation/governance/blob/main/documents/OMF-MDS-Privacy-Guide-for-Cities.pdf).
262+
263+
[Top][toc]
264+
265+
### Special Group Type
266+
267+
Here are the possible values for the `special_group_type` dimension field:
268+
269+
| Name | Description |
270+
| ---------------- | --------------------------------------------------------------------------------------------------------------------- |
271+
| low_income | Trips where a low income discount is applied by the provider, e.g., a discount from a qualified provider equity plan. |
272+
| adaptive_scooter | Trips taken on a scooter with features to improve accessibility for people with disabilities, e.g., scooter with a seat or wider base |
273+
| all_riders | All riders from any group |
274+
275+
Other special group types may be added in future MDS releases as relevant agency and provider use cases are identified. When additional special group types or metrics are proposed, a thorough review of utility and relevance in program oversight, evaluation, and policy development should be done by OMF Working Groups, as well as any privacy implications by the OMF Privacy Committee.
276+
277+
[Top][toc]
278+
225279
[agency]: /agency/README.md
226280
[accessibility-options]: /modes/README.md#accessibility-options
227281
[decimal-degrees]: https://en.wikipedia.org/wiki/Decimal_degrees

provider/README.md

Lines changed: 4 additions & 107 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@ This specification contains a data standard for *mobility as a service* provider
3232
* [Reports](#reports)
3333
* [Reports - Response](#reports---response)
3434
* [Reports - Example](#reports---example)
35-
* [Special Group Type](#special-group-type)
3635
* [Data Redaction](#data-redaction)
3736
* [Realtime Data](#realtime-data)
3837
* [Data Latency Requirements][data-latency]
@@ -342,116 +341,17 @@ The authenticated reports are monthly, historic flat files that may be pre-gener
342341

343342
**Endpoint:** `/reports`
344343
**Method:** `GET`
345-
**[Beta feature][beta]:** Yes (as of 1.1.0). [Leave feedback](https://github.com/openmobilityfoundation/mobility-data-specification/issues/672)
344+
**[Beta feature][beta]:** No (as of 2.0.0). [Leave feedback](https://github.com/openmobilityfoundation/mobility-data-specification/issues/672)
346345
**Usage note:** This endpoint uses media-type `text/vnd.mds+csv` instead of `application/vnd.mds+json`, see [Versioning][versioning].
347-
**Schema:** TBD when out of beta
346+
**Schema:** TBD
348347
**`data` Filename:** monthly file named by year and month, e.g. `/reports/YYYY-MM.csv`
349-
**`data` Payload:** monthly CSV files with the following structure:
350-
351-
| Column Name | Type | Comments |
352-
|----------------------| ----------------------------------------- | ------------------------------------------------ |
353-
| `provider_id` | UUID | A UUID for the Provider, unique within MDS. See MDS provider_id in [provider list](/providers.csv). |
354-
| `start_date` | date | Start date of trip the data row, ISO 8601 date format, i.e. YYYY-MM-DD |
355-
| `duration` | string | Value is always `P1M` for monthly. Based on [ISO 8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations) |
356-
| `special_group_type` | [Special Group Type](#special-group-type) | Type that applies to this row |
357-
| `geography_id` | [Geography](/geography) | ID that applies to this row. Includes all IDs in /geography. When there is no /geography then return `null` for this value and return counts based on the entire operating area. |
358-
| `vehicle_type` | [Vehicle Type](/agency#vehicle-type) | Type that applies to this row |
359-
| `trip_count` | integer | Count of trips taken for this row |
360-
| `rider_count` | integer | Count of unique riders for this row |
361-
362-
#### Data Notes
363-
364-
Report contents include every combination of special group types, geography IDs, and vehicle types in operation for each month since the provider began operations in the jurisdiction. New files are added monthly in addition to the previous monthly historic files.
365-
366-
Counts are calculated based the agency's local time zone. Trips are counted based on their start time, i.e. if a trip starts in month A but ends in month B, it will be counted only as part of the report for month A. Similarly, trips are counted based on their start geography, i.e. if a trip starts in geography A and ends in geography B, it will appear in the counts for geography A and not for geography B.
367-
368-
All geography IDs included in the city published [Geography](/geography) API endpoint are included in the report results. In lieu of serving an API, this can alternately be a [flat file](/geography#file-format) created by the city and sent to the provider via link. If there is no `/geography` available, then counts are for the entire agency operating area, and `null` is returned for each Geography ID.
348+
**`data` Payload:** monthly CSV files of [Report](/data-types.md#Reports) objects
369349

370350
[Top][toc]
371351

372352
### Reports - Example
373353

374-
For 3 months of provider operation in a city (September 2019 through November 2019) for 3 geographies, 2 vehicle types, and 1 special group. Values of `-1` represent [redacted data](#data-redaction) counts.
375-
376-
**September 2019** `/reports/2019-09.csv`
377-
378-
```csv
379-
provider_id,start_date,duration,special_group_type,geography_id,vehicle_type,trip_count,rider_count
380-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,scooter,1302,983
381-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,scooter,201,104
382-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,530,200
383-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,75,26
384-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,687,450
385-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,98,45
386-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,256,104
387-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,41,16
388-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,201,140
389-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,35,21
390-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,103,39
391-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-09-01,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,15,-1
392-
```
393-
394-
**October 2019** `/reports/2019-10.csv`
395-
396-
```csv
397-
provider_id,start_date,duration,special_group_type,geography_id,vehicle_type,trip_count,rider_count
398-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,scooter,1042,786
399-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,scooter,161,83
400-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,424,160
401-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,60,0
402-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,550,360
403-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,78,36
404-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,205,83
405-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,33,13
406-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,161,112
407-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,28,-1
408-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,82,31
409-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-10-01,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,-1,0
410-
```
411-
412-
**November 2019** `/reports/2019-11.csv`
413-
414-
```csv
415-
provider_id,start_date,duration,special_group_type,geography_id,vehicle_type,trip_count,rider_count
416-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,scooter,834,629
417-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,scooter,129,66
418-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,339,128
419-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,48,-1
420-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,440,288
421-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,62,29
422-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,164,66
423-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,26,0
424-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,129,90
425-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,22,-1
426-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,-1,25
427-
48415839-3e38-4ba5-a557-e45fb4e6a0a3,2019-11-01,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,0,0
428-
```
429-
430-
[Top][toc]
431-
432-
### Special Group Type
433-
434-
Here are the possible values for the `special_group_type` dimension field:
435-
436-
| Name | Description |
437-
| ---------------- | --------------------------------------------------------------------------------------------------------------------- |
438-
| low_income | Trips where a low income discount is applied by the provider, e.g., a discount from a qualified provider equity plan. |
439-
| adaptive_scooter | Trips taken on a scooter with features to improve accessibility for people with disabilities, e.g., scooter with a seat or wider base |
440-
| all_riders | All riders from any group |
441-
442-
Other special group types may be added in future MDS releases as relevant agency and provider use cases are identified. When additional special group types or metrics are proposed, a thorough review of utility and relevance in program oversight, evaluation, and policy development should be done by OMF Working Groups, as well as any privacy implications by the OMF Privacy Committee.
443-
444-
[Top][toc]
445-
446-
### Data Redaction
447-
448-
Some combinations of parameters may return a small count of trips, which could increase a privacy risk of re-identification. To correct for that, Reports does not return data below a certain count of results. This data redaction is called k-anonymity, and the threshold is set at a k-value of 10. For more explanation of this methodology, see our [Data Redaction Guidance document](https://github.com/openmobilityfoundation/mobility-data-specification/wiki/MDS-Data-Redaction).
449-
450-
**If the query returns fewer than `10` trips in a count, then that row's count value is returned as "-1".** Note "0" values are also returned as "-1" since the goal is to group both low and no count values for privacy.
451-
452-
As Reports is in [beta][beta], this value may be adjusted in future releases and/or may become dynamic to account for specific categories of use cases and users. To improve the specification and to inform future guidance, beta users are encouraged to share their feedback and questions about k-values on this [discussion thread](https://github.com/openmobilityfoundation/mobility-data-specification/discussions/622).
453-
454-
Using k-anonymity will reduce, but not necessarily eliminate the risk that an individual could be re-identified in a dataset, and this data should still be treated as sensitive. This is just one part of good privacy protection practices, which you can read more about in our [MDS Privacy Guide for Cities](https://github.com/openmobilityfoundation/governance/blob/main/documents/OMF-MDS-Privacy-Guide-for-Cities.pdf).
354+
See [Provider examples](examples.md#reports).
455355

456356
[Top][toc]
457357

@@ -505,7 +405,6 @@ Unless stated otherwise by the municipality, this endpoint must return only thos
505405
506406
**Endpoint:** `/telemetry`
507407
**Method:** `GET`
508-
**[Beta feature][beta]:** No (as of 2.0.0)
509408
**Schema:** [`telemetry` schema][telemetry-schema]
510409
**`data` Payload:** `{ "telemetry": [] }`, an array of `telemetry` objects
511410

@@ -523,8 +422,6 @@ Without a `telemetry_time` query parameter, `/telemetry` shall return a `400 Bad
523422

524423
The `/vehicles` is a near-realtime endpoint and returns the current status of vehicles in an agency's [Jurisdiction](/general-information.md#definitions) and/or area of agency responsibility. All vehicles that are currently in any [`vehicle_state`][vehicle-states] should be returned in this payload. Since all states are returned, care should be taken to filter out states not in the [PROW](/general-information.md#definitions) if doing vehicle counts. For the states `elsewhere` and `removed` which include vehicles not in the [PROW](/general-information.md#definitions) but provide some operational clarity for agencies, these must only persist in the feed for 90 minutes before being removed.
525424

526-
Data in this endpoint should reconcile with data from the historic [`/status_changes`](/provider#events) endpoint, though `/events` is the source of truth if there are discrepancies.
527-
528425
As with other MDS APIs, `/vehicles` is intended for use by regulators, not by the general public. `/vehicles` can be deployed by providers as a standalone MDS endpoint for agencies without requiring the use of other endpoints, due to the [modularity](/README.md#modularity) of MDS. See our [MDS Vehicles Guide](https://github.com/openmobilityfoundation/mobility-data-specification/wiki/MDS-Vehicles) for how this compares to GBFS `/free_bike_status`. Note that using authenticated `/vehicles` does not replace the role of a public [GBFS][gbfs] feed in enabling consumer-facing applications. If a provider is using both `/vehicles` and GBFS endpoints, the `/vehicles` endpoint should be considered source of truth regarding an agency's compliance checks.
529426

530427
In addition to the standard [Provider payload wrapper](#response-format), responses from this endpoint should contain the last update timestamp and amount of time until the next update in accordance with the [Data Latency Requirements][data-latency]:

0 commit comments

Comments
 (0)