Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Add MODIFIED and pickup and drop-off types to GTFS-RT; experimentally deprecate SKIPPED #265

Closed
wants to merge 17 commits into from
+161 −5
Closed

Proposal: Add MODIFIED and pickup and drop-off types to GTFS-RT; experimentally deprecate SKIPPED #265

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
2faa3df
Add pick-up and drop-off types
ericouyang Mar 23, 2021
b28420b
Update documentation
ericouyang Mar 23, 2021
898c1a3
Document TripUpdates V3
ericouyang Mar 26, 2021
9a57863
Revert "Document TripUpdates V3"
ericouyang Aug 24, 2021
bf8bfa7
Add migration approach
ericouyang Aug 25, 2021
591ddd8
Fix typo
ericouyang Aug 25, 2021
16e412f
Update language to adhere to RFC 2119
ericouyang Aug 25, 2021
c475594
Address PR feedback
ericouyang Aug 25, 2021
89993cc
Mark SKIPPED as deprecated
ericouyang Aug 25, 2021
19fc85f
Update migration guide
ericouyang Aug 26, 2021
107fb2a
Merge branch 'master' into gtfs-servicechanges-v3.1-pickupdropofftypes
ericouyang Aug 26, 2021
780be36
Editorial clarifications
ericouyang Sep 28, 2021
e9ef94b
Reflect editorial changes into reference.md
ericouyang Sep 28, 2021
ab21cd6
No longer mark SKIPPED as formally deprecated
ericouyang Oct 7, 2021
81c94d7
Update migration guide to support SCHEDULED
ericouyang Oct 7, 2021
4ffaf3a
Support SCHEDULED in migration period
ericouyang Oct 7, 2021
d28f432
Update reference to match documentation in .proto
ericouyang Oct 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions gtfs-realtime/proto/gtfs-realtime.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,14 @@ message TripUpdate {
// NOTE: This field is still experimental, and subject to change. It may be
// formally adopted in the future.
UNSCHEDULED = 3;

// A schedule-based vehicle's StopTimeProperties differs from the static schedule in a way that impacts
// the rider experience, such as a vehicle not picking up passengers because a vehicle is "drop-off only"
// to catch up back to schedule when it's running late. StopTimeProperties must included to specify how
// the relationship has been modified.
// NOTE: This field is still experimental, and subject to change. It may be
// formally adopted in the future.
MODIFIED = 4;
ericouyang marked this conversation as resolved.
Show resolved Hide resolved
ericouyang marked this conversation as resolved.
Show resolved Hide resolved
}
optional ScheduleRelationship schedule_relationship = 5
[default = SCHEDULED];
Expand All @@ -260,6 +268,54 @@ message TripUpdate {
// NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
optional string assigned_stop_id = 1;

// Supports real-time changes to pickup_type when it differs from what's specified in (CSV) GTFS
// in stop_times.txt. See definition of stop_times.pickup_type in (CSV) GTFS.
// If pickup_type is specified, StopTimeUpdate.schedule_relationship must be MODIFIED for schedule-based service.
// NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
optional PickupType pickup_type = 2;

// Supports real-time changes to drop_off_type when it differs from what's specified in (CSV) GTFS
// in stop_times.txt. See definition of stop_times.drop_off_type in (CSV) GTFS.
// If drop_off_type is specified, StopTimeUpdate.schedule_relationship must be MODIFIED for scheduled-based service.
// NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
optional DropOffType drop_off_type = 3;

// Indicates the pickup method for an updated stop time when it differs from what's specified in (CSV) GTFS
// in stop_times.txt. By default, if pickup_type is neither specified in real-time or in (CSV) GTFS, it is
// considered REGULAR_PICKUP.
// NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
enum PickupType {
// See definition of stop_times.pickup_type=0 in (CSV) GTFS.
REGULAR_PICKUP = 0;

// See definition of stop_times.pickup_type=1 in (CSV) GTFS.
NO_PICKUP = 1;

// See definition of stop_times.pickup_type=2 in (CSV) GTFS.
MUST_PHONE_AGENCY_PICKUP = 2;

// See definition of stop_times.pickup_type=3 in (CSV) GTFS.
MUST_ASK_DRIVER_PICKUP = 3;
}

// Indicates the drop off method for an updated stop time when it differs from what's specified in (CSV) GTFS
// in stop_times.txt. By default, if drop_off_type is neither specified in real-time or in (CSV) GTFS, it is
// considered REGULAR_DROP_OFF.
// NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
enum DropOffType {
// See definition of stop_times.drop_off_type=0 in (CSV) GTFS.
REGULAR_DROP_OFF = 0;

// See definition of stop_times.drop_off_type=1 in (CSV) GTFS.
NO_DROP_OFF = 1;

// See definition of stop_times.drop_off_type=2 in (CSV) GTFS.
MUST_PHONE_AGENCY_DROP_OFF = 2;

// See definition of stop_times.drop_off_type=3 in (CSV) GTFS.
MUST_ASK_DRIVER_DROP_OFF = 3;
}

// The extensions namespace allows 3rd-party developers to extend the
// GTFS Realtime Specification in order to add and evaluate new features
// and modifications to the spec.
Expand Down
40 changes: 40 additions & 0 deletions gtfs-realtime/spec/en/examples/migration-modified.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
## Migration Guide - Transition from SKIPPED to MODIFIED

The GTFS-realtime `StopTimeUpdate.schedule_relationship` of `SKIPPED` indicates that a vehicle will not be serving a particular stop that was originally scheduled as a part of `stop_times.txt` in GTFS-static.

The GTFS-static `pickup_type` and `drop_off_type` fields within `stop_times.txt` allow an agency to specify details on what a rider needs to do in order to board/alight at a given stop (e.g. a rider must explicitly request a drop-off at a stop, otherwise the driver will skip the stop).

This migration guide defines how existing producers who use the `SKIPPED` enumeration as a best approximation of partial service at a stop can provide a more nuanced picture of the level of service through `pickup_type` and `drop_off_type` via `StopTimeProperties`. The goal is to minimize disruption to producers and consumers during the transition.

See the [Add pickup and drop-off types to GTFS-RT proposal on GitHub](https://github.com/google/transit/pull/265).

### Initially providing SKIPPED and pickup_type / drop_off_type in same feed

#### Producers

If you are a producer who uses the `SKIPPED` enumeration as a best approximation of partial service at a stop, to avoid disruption to existing consumers, it is recommended that you continue to produce `SKIPPED` entities for those stop time updates but also add `pickup_type` and `drop_off_type` entities for the same `StopTimeUpdate`.

Here's an example of indicating to consumers that a vehicle will be "drop-off only":

~~~
entity {
id: "ei0"
trip_update {
...
stop_time_update {
...
schedule_relationship: SKIPPED

stop_time_properties {
pickup_type: NO_PICKUP
drop_off_type: MUST_ASK_DRIVER_DROP_OFF
}
}
}
}
~~~

It is suggested that you notify existing consumers (e.g., via a developer mailing list) that the use of `SKIPPED` for partial service at a stop is being deprecated in favor of more nuanced information by a set deadline and that consumers should start consuming the `MODIFIED` enumeration as well as `pickup_type` and `drop_off_type`. You should also provide a link to this migration guide. After the deadline passes, you can stop using `SKIPPED` entities for scenarios where `pickup_type` or `drop_off_type` are included and start using `MODIFIED` instead.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you're effectively deprecating SKIPPED? If so, please mark it as deprecated in the .proto with [deprecated=true] and docs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally, I was thinking that the spec would continue to support SKIPPED for service that's fully missed, such as due to a stop closure. That said, there is ambiguity in that scenario in terms of whether or not you would do SKIPPED or MODIFIED with pickup_type=NO_PICKUP and drop_off_type=NO_DROP_OFF.

Do you have a suggestion in terms of whether or not deprecation makes sense or instead to specify that producers should not use both NO_PICKUP and NO_DROP_OFF at the same time and continue to use SKIPPED?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SKIPPED is currently defined as:

The stop is skipped, i.e., the vehicle will not stop at this stop.

Without additional context, I think SKIPPED and pickup_type=NO_PICKUP with drop_off_type=NO_DROP_OFF are effectively the same case from a consumers perspective (but correct me if I'm wrong). If that's true, I'd argue that we should deprecate SKIPPED in favor of additional granularity in pickup_type and drop_off_type so there is one clear way of representing this information going forward.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm misunderstanding, deprecating SKIPPED require producers and consumers to stop using it and instead use pickup_type=NO_PICKUP with drop_off_type=NO_DROP_OFF. Eric's suggestion "to specify that producers should not use both NO_PICKUP and NO_DROP_OFF at the same time and continue to use SKIPPED" seems like a better approach.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm misunderstanding, deprecating SKIPPED require producers and consumers to stop using it and instead use pickup_type=NO_PICKUP with drop_off_type=NO_DROP_OFF. Eric's suggestion "to specify that producers should not use both NO_PICKUP and NO_DROP_OFF at the same time and continue to use SKIPPED" seems like a better approach.

Not immediately - the long-term plan would be to stop using SKIPPED (i.e., it's deprecated, signaling that at some point it will be removed entirely from the spec and .proto), but there needs to be a short term migration plan (like Eric's suggestion to use both in parallel for some time) because it will take consumers and producers time to switch over. Using [deprecated=true] in the .proto allows producers and consumers to still continue using that field in the bindings software for now, it will just mark the relevant functions as deprecated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deprecating SKIPPED and using the proposed migration plan sounds good in theory, but we do have concerns about this in practice. There are a lot of producers and consumers who would need to make updates to stop using SKIPPED, and we could see the [deprecated=true] sticking around for a very long time. Would there be a point at which the field would be removed altogether?

Another practical concern is that the migration plan makes suggestions instead of requirements.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely agree here about the practical considerations of deprecating fields. I think it's fair to say that if we were designing the spec from scratch, we probably wouldn't have both SKIPPED and NO_PICKUP and NO_DROP_OFF to avoid the ambiguity of representing the same scenario two different ways. Even when documented, if it's technically allowed, it creates room for confusion.

I would expect it to take quite a while for both consumers and producers to update (perhaps years), but I think signally it now by marking it explicitly as deprecated allows us the ability to eventually clean up the ambiguity.


#### Consumers
`pickup_type` and `drop_off_type` set in `StopTimeProperties` takes precedence over `schedule_relationship=SKIPPED`, meaning that real-time arrival information should still be used to inform passengers rather being ignored, perhaps with additional instructions or notification on how to board/alight, similar to what may happen with those fields being specified in `stop_times.txt`.
38 changes: 36 additions & 2 deletions gtfs-realtime/spec/en/reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,8 +184,8 @@ The update is linked to a specific stop either through stop_sequence or stop_id,
|------------------|------------|----------------|-------------------|-------------------|
| **stop_sequence** | [uint32](https://developers.google.com/protocol-buffers/docs/proto#scalar) | Conditionally required | One | Must be the same as in stop_times.txt in the corresponding GTFS feed. Either stop_sequence or stop_id must be provided within a StopTimeUpdate - both fields cannot be empty. stop_sequence is required for trips that visit the same stop_id more than once (e.g., a loop) to disambiguate which stop the prediction is for. If `StopTimeProperties.assigned_stop_id` is populated, then `stop_sequence` must be populated. |
| **stop_id** | [string](https://developers.google.com/protocol-buffers/docs/proto#scalar) | Conditionally required | One | Must be the same as in stops.txt in the corresponding GTFS feed. Either stop_sequence or stop_id must be provided within a StopTimeUpdate - both fields cannot be empty. If `StopTimeProperties.assigned_stop_id` is populated, it is preferred to omit `stop_id` and use only `stop_sequence`. If `StopTimeProperties.assigned_stop_id` and `stop_id` are populated, `stop_id` must match `assigned_stop_id`. |
| **arrival** | [StopTimeEvent](#message-stoptimeevent) | Conditionally required | One | If schedule_relationship is empty or SCHEDULED, either arrival or departure must be provided within a StopTimeUpdate - both fields cannot be empty. arrival and departure may both be empty when schedule_relationship is SKIPPED. If schedule_relationship is NO_DATA, arrival and departure must be empty. |
| **departure** | [StopTimeEvent](#message-stoptimeevent) | Conditionally required | One | If schedule_relationship is empty or SCHEDULED, either arrival or departure must be provided within a StopTimeUpdate - both fields cannot be empty. arrival and departure may both be empty when schedule_relationship is SKIPPED. If schedule_relationship is NO_DATA, arrival and departure must be empty. |
| **arrival** | [StopTimeEvent](#message-stoptimeevent) | Conditionally required | One | If schedule_relationship is empty or SCHEDULED, either arrival or departure must be provided within a StopTimeUpdate - both fields cannot be empty. arrival and departure may both be empty when schedule_relationship is SKIPPED or if only `StopTimeProperties` is being updated when schedule_relationship is MODIFIED but no real-time timing is available. If schedule_relationship is NO_DATA, arrival and departure must be empty. |
| **departure** | [StopTimeEvent](#message-stoptimeevent) | Conditionally required | One | If schedule_relationship is empty or SCHEDULED, either arrival or departure must be provided within a StopTimeUpdate - both fields cannot be empty. arrival and departure may both be empty when schedule_relationship is SKIPPED or if only `StopTimeProperties` is being updated when schedule_relationship is MODIFIED but no real-time timing is available. If schedule_relationship is NO_DATA, arrival and departure must be empty. |
| **schedule_relationship** | [ScheduleRelationship](#enum-schedulerelationship) | Optional | One | The default relationship is SCHEDULED. |
| **stop_time_properties** | [StopTimeProperties](#message-stoptimeproperties) | Optional | One | Realtime updates for certain properties defined within GTFS stop_times.txt <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. |

Expand All @@ -201,6 +201,7 @@ The relation between this StopTime and the static schedule.
| **SKIPPED** | The stop is skipped, i.e., the vehicle will not stop at this stop. Arrival and departure are optional. When set `SKIPPED` is not propagated to subsequent stops in the same trip (i.e., the vehicle will stop at subsequent stops in the trip unless those stops also have a `stop_time_update` with `schedule_relationship: SKIPPED`). Delay from a previous stop in the trip *does* propagate over the `SKIPPED` stop. In other words, if a `stop_time_update` with an `arrival` or `departure` prediction is not set for a stop after the `SKIPPED` stop, the prediction upstream of the `SKIPPED` stop will be propagated to the stop after the `SKIPPED` stop and subsequent stops in the trip until a `stop_time_update` for a subsequent stop is provided. |
| **NO_DATA** | No data is given for this stop. It indicates that there is no realtime information available. When set NO_DATA is propagated through subsequent stops so this is the recommended way of specifying from which stop you do not have realtime information. When NO_DATA is set neither arrival nor departure should be supplied. |
| **UNSCHEDULED** | The vehicle is operating a frequency-based trip (GTFS frequencies.txt with exact_times = 0). This value should not be used for trips that are not defined in GTFS frequencies.txt, or trips in GTFS frequencies.txt with exact_times = 1. Trips containing `stop_time_updates` with `schedule_relationship: UNSCHEDULED` must also set the TripDescriptor `schedule_relationship: UNSCHEDULED` <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future.
| **MODIFIED** | The vehicle's relationship with the static schedule differs in a way that impacts the rider experience, such as a vehicle not picking up passengers because a vehicle is "drop-off only" to catch up back to schedule when it's running late. StopTimeProperties must included to specify how the relationship has been modified. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future.

## _message_ StopTimeProperties

Expand All @@ -213,6 +214,39 @@ Realtime update for certain properties defined within GTFS stop_times.txt.
| _**Field Name**_ | _**Type**_ | _**Required**_ | _**Cardinality**_ | _**Description**_ |
|------------------|------------|----------------|-------------------|-------------------|
| **assigned_stop_id** | [string](https://developers.google.com/protocol-buffers/docs/proto#scalar) | Optional | One | Supports real-time stop assignments. Refers to a `stop_id` defined in the GTFS `stops.txt`. <br> The new `assigned_stop_id` should not result in a significantly different trip experience for the end user than the `stop_id` defined in GTFS `stop_times.txt`. In other words, the end user should not view this new `stop_id` as an "unusual change" if the new stop was presented within an app without any additional context. For example, this field is intended to be used for platform assignments by using a `stop_id` that belongs to the same station as the stop originally defined in GTFS `stop_times.txt`. <br> To assign a stop without providing any real-time arrival or departure predictions, populate this field and set `StopTimeUpdate.schedule_relationship = NO_DATA`. <br> If this field is populated, `StopTimeUpdate.stop_sequence` must be populated and `StopTimeUpdate.stop_id` should not be populated. Stop assignments should be reflected in other GTFS-realtime fields as well (e.g., `VehiclePosition.stop_id`). <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. |
| **pickup_type** | [PickupType](#enum-pickuptype) | Optional | One | Supports real-time changes to pickup_type when it differs from what's specified in (CSV) GTFS in `stop_times.txt`. See definition of `stop_times.pickup_type` in (CSV) GTFS. If `pickup_type` is specified, `StopTimeUpdate.schedule_relationship` must be `MODIFIED` for schedule-based service. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. |
| **drop_off_type** | [DropOffType](#enum-dropofftype) | Optional | One | Supports real-time changes to drop_off_type when it differs from what's specified in (CSV) GTFS in `stop_times.txt`. See definition of `stop_times.drop_off_type` in (CSV) GTFS. If drop_off_type is specified, `StopTimeUpdate.schedule_relationship` must be `MODIFIED` for schedule-based service.
<br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. |

## _enum_ PickupType

Indicates the pickup method for an updated stop time when it differs from what's specified in (CSV) GTFS in stop_times.txt By default, if pickup_type is neither specified in real-time or in (CSV) GTFS, it is considered REGULAR_PICKUP.

**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future.

#### Values

| _**Value**_ | _**Comment**_ |
|-------------|---------------|
| **REGULAR_PICKUP** | See definition of stop_times.pickup_type=0 in (CSV) GTFS. |
| **NO_PICKUP** | See definition of stop_times.pickup_type=1 in (CSV) GTFS. |
| **MUST_PHONE_AGENCY_PICKUP** | See definition of stop_times.pickup_type=2 in (CSV) GTFS. |
| **MUST_ASK_DRIVER_PICKUP** | See definition of stop_times.pickup_type=3 in (CSV) GTFS. |

## _enum_ DropOffType

Indicates the drop off method for an updated stop time when it differs from what's specified in (CSV) GTFS in stop_times.txt. By default, if drop_off_type is neither specified in real-time or in (CSV) GTFS, it is considered REGULAR_DROP_OFF.

**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future.

#### Values

| _**Value**_ | _**Comment**_ |
|-------------|---------------|
| **REGULAR_DROP_OFF** | See definition of stop_times.drop_off_type=0 in (CSV) GTFS. |
| **NO_DROP_OFF** | See definition of stop_times.drop_off_type=1 in (CSV) GTFS. |
| **MUST_PHONE_AGENCY_DROP_OFF** | See definition of stop_times.drop_off_type=2 in (CSV) GTFS. |
| **MUST_ASK_DRIVER_DROP_OFF** | See definition of stop_times.drop_off_type=3 in (CSV) GTFS. |

## _message_ TripProperties

Expand Down
4 changes: 2 additions & 2 deletions gtfs-realtime/spec/en/trip-updates.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ Each [StopTimeUpdate](reference.md#StopTimeUpdate) is linked to a stop. Ordinari

The update can provide a exact timing for **arrival** and/or **departure** at a stop in [StopTimeUpdates](reference.md#StopTimeUpdate) using [StopTimeEvent](reference.md#StopTimeEvent). This should contain either an absolute **time** or a **delay** (i.e. an offset from the scheduled time in seconds). Delay can only be used in case the trip update refers to a scheduled GTFS trip, as opposed to a frequency-based trip. In this case, time should be equal to scheduled time + delay. You may also specify **uncertainty** of the prediction along with [StopTimeEvent](reference.md#StopTimeEvent), which is discussed in more detail in section [Uncertainty](#uncertainty) further down the page.

For each [StopTimeUpdate](reference.md#StopTimeUpdate), the default schedule relationship is **scheduled**. (Note that this is different from the schedule relationship for the trip). You may change this to **skipped** if the stop will not be stopped at, or **no data** if you only have realtime data for some of the trip.
For each [StopTimeUpdate](reference.md#StopTimeUpdate), the default schedule relationship is **scheduled**. (Note that this is different from the schedule relationship for the trip). You may change this to **skipped** if the stop will not be stopped at, **modified** if stop time properties are different than what's specified in `stop_times.txt`, or **no data** if you only have realtime data for some of the trip.

**Updates should be sorted by stop_sequence** (or stop_ids in the order they occur in the trip).

If one or more stops are missing along the trip the `delay` from the update (or, if only `time` is provided in the update, a delay computed by comparing the `time` against the GTFS schedule time) is propagated to all subsequent stops. This means that updating a stop time for a certain stop will change all subsequent stops in the absence of any other information. Note that updates with a schedule relationship of `SKIPPED` will not stop delay propagation, but updates with schedule relationships of `SCHEDULED` (also the default value if schedule relationship is not provided) or `NO_DATA` will.
If one or more stops are missing along the trip the `delay` from the update (or, if only `time` is provided in the update, a delay computed by comparing the `time` against the GTFS schedule time) is propagated to all subsequent stops. This means that updating a stop time for a certain stop will change all subsequent stops in the absence of any other information. Note that updates with a schedule relationship of `SKIPPED` or `MODIFIED` will not stop delay propagation, but updates with schedule relationships of `SCHEDULED` (also the default value if schedule relationship is not provided) or `NO_DATA` will.

**Example 1**

Expand Down