Implemented rules

March 23, 2021 · View on GitHub

Notices related to file parsing and data types are defined in the core
Notices related to GTFS semantics/business logic are encapsulated within the related validation rule class see the following example in TripUsageValidator:

/**
 * Validates that every trip in "trips.txt" is used by some stop from "stop_times.txt"
 *
 * <p>Generated notice: {@link UnusedTripNotice}.
 */
@GtfsValidator
public class TripUsageValidator extends FileValidator {
  private final GtfsTripTableContainer tripTable;
  private final GtfsStopTimeTableContainer stopTimeTable;

  @Inject
  TripUsageValidator(GtfsTripTableContainer tripTable, GtfsStopTimeTableContainer stopTimeTable) {
    this.tripTable = tripTable;
    this.stopTimeTable = stopTimeTable;
  }

  @Override
  public void validate(NoticeContainer noticeContainer) {
    // Do not report the same trip_id multiple times.
    Set<String> reportedTrips = new HashSet<>();
    for (GtfsTrip trip : tripTable.getEntities()) {
      String tripId = trip.tripId();
      if (reportedTrips.add(tripId) && stopTimeTable.byTripId(tripId).isEmpty()) {
        noticeContainer.addValidationNotice(new UnusedTripNotice(tripId, trip.csvRowNumber()));
      }
    }
  }
  /**
   * A {@code GtfsTrip} should be referred to at least once in {@code GtfsStopTimeTableContainer}
   * station).
   *
   * <p>Severity: {@code SeverityLevel.WARNING}
   */
  static class UnusedTripNotice extends ValidationNotice {
    UnusedTripNotice(String tripId, long csvRowNumber) {
      super(
          ImmutableMap.of(
              "tripId", tripId,
              "csvRowNumber", csvRowNumber),
          SeverityLevel.WARNING);
    }
  }
}

Note that the notice ID naming conventions changed in v2 to make contributions of new rules easier by reducing the likelihood of conflicting IDs during parallel development. Please refer to MIGRATION_V1_V2.md for a mapping between v1 and v2 rules.

Definitions

Notices are split into three categories: INFO, WARNING, ERROR.

ERROR notices are for items that the GTFS reference specification explicitly requires or prohibits (e.g., using the language "must"). The validator uses RFC2119 to interpret the language in the GTFS spec.
- ⚠️ for this particular level of severity, ValidationNotices should be distinguished from SystemErrors: while ValidationNotices give information about the data quality, SystemErrors are not semantic errors, they give information about things that may have gone wrong during the validation process such as an impossibility to unzip a GTFS archive.
WARNING notices are for items that will affect the quality of GTFS datasets but the GTFS spec does expressly require or prohibit. For example, these might be items recommended using the language "should" or "should not" in the GTFS spec, or items recommended in the MobilityData GTFS Best Practices.
INFO notices are for items that do not affect the feed's quality, such as unknown files or unknown fields.

Table of errors

Name	Description
`BlockTripsWithOverlappingStopTimesNotice`	Block trips with overlapping stop times.
`DecreasingOrEqualShapeDistanceNotice`	Decreasing or equal `shape_dist_traveled` in `shapes.txt`.
`DecreasingOrEqualStopTimeDistanceNotice`	Decreasing or equal `shape_dist_traveled` in `stop_times.txt`.
`DuplicatedColumnNotice`	Duplicated column in CSV.
`DuplicateFareRuleZoneIdFieldsNotice`	Duplicate rows rows from `fare_rules.txt` based on `fare_rules.route_id`, `fare_rules.origin_id`, `fare_rules.contains_id` and `fare_rules.destination_id`.
`DuplicateKeyNotice`	Duplicated entity.
`EmptyFileNotice`	A CSV file is empty.
`ForeignKeyViolationNotice`	Wrong foreign key.
`InconsistentAgencyTimezoneNotice`	Inconsistent Timezone among agencies.
`InvalidColorNotice`	A field contains an invalid color value.
`InvalidCurrencyNotice`	A field contains a wrong currency code.
`InvalidDateNotice`	A field cannot be parsed as date.
`InvalidEmailNotice`	A field contains a malformed email address.
`InvalidFloatNotice`	A field cannot be parsed as a floating point number.
`InvalidIntegerNotice`	A field cannot be parsed as an integer.
`InvalidLanguageCodeNotice`	A field contains a wrong language code.
`InvalidPhoneNumberNotice`	A field contains a malformed phone number.
`InvalidRowLengthNotice`	Invalid csv row length.
`InvalidTimeNotice`	A field cannot be parsed as time.
`InvalidTimezoneNotice`	A field cannot be parsed as a timezone.
`InvalidUrlNotice`	A field contains a malformed URL.
`LeadingOrTrailingWhitespacesNotice`	The value in CSV file has leading or trailing whitespaces.
`LocationWithoutParentStationNotice`	A location that must have `parent_station` field does not have it.
`MissingCalendarAndCalendarDateFilesNotice`	Missing GTFS files `calendar.txt` and `calendar_dates.txt`.
`MissingRequiredColumnNotice`	A required column is missing in the input file.
`MissingRequiredFieldNotice`	A required field is missing.
`MissingRequiredFileNotice`	A required file is missing.
`MissingTripEdgeNotice`	Missing trip edge `arrival_time` or `departure_time`.
`NewLineInValueNotice`	New line or carriage return in a value in CSV file.
`NumberOutOfRangeNotice`	Out of range value.
`OverlappingFrequencyNotice`	Trip frequencies overlap.
`RouteBothShortAndLongNameMissingNotice`	Missing route short name and long name.
`SameNameAndDescriptionForRouteNotice`	Same name and description for route.
`StartAndEndRangeEqualNotice`	Two date or time fields are equal.
`StartAndEndRangeOutOfOrderNotice`	Two date or time fields are out of order.
`StationWithParentStationNotice`	A station has `parent_station` field set.
`StopTimeWithArrivalBeforePreviousDepartureTimeNotice`	Backwards time travel between stops in `stop_times.txt`
`StopTimeWithOnlyArrivalOrDepartureTimeNotice`	Missing `stop_times.arrival_time` or `stop_times.departure_time`.
`WrongParentLocationTypeNotice`	Incorrect type of the parent location.

Table of warnings

Name	Description
`AttributionWithoutRoleNotice`	Attribution with no role.
`DuplicateRouteNameNotice`	Duplicate `routes.route_long_name`. Duplicate `routes.route_short_name`. Duplicate combination of fields `route_long_name` and `routes.route_short_name`.
`EmptyColumnNameNotice`	A column name is empty.
`EmptyRowNotice`	A file is unknown.
`FeedExpirationDateNotice`	Dataset should be valid for at least the next 7 days. Dataset should cover at least the next 30 days of service.
`FeedInfoLangAndAgencyMismatchNotice`	Mismatching feed and agency language fields.
`InconsistentAgencyLangNotice`	Inconsistent language among agencies.
`MissingFeedInfoDateNotice`	`feed_end_date` should be provided if `feed_start_date` is provided. `feed_start_date` should be provided if `feed_end_date` is provided.
`MoreThanOneEntityNotice`	More than one row in CSV.
`NonAsciiOrNonPrintableCharNotice`	Non ascii or non printable char in `id`.
`PlatformWithoutParentStationNotice`	A platform has no `parent_station` field set.
`RouteColorContrastNotice`	Insufficient route color contrast.
`RouteShortAndLongNameEqualNotice`	Short and long name are equal for a route.
`RouteShortNameTooLongNotice`	Short name of a route is too long (more than 12 characters).
`StartAndEndTimeEqualNotice`	Equal `frequencies.start_time` and `frequencies.end_time`.
`StopTimeTimepointWithoutTimesNotice`	`arrival_time` or `departure_time` not specified for timepoint.
`StopTooFarFromTripShapeNotice`	Stop too far from trip shape.
`TooFastTravelNotice`	Fast travel between stops in `stop_times.txt`.
`UnexpectedEnumValueNotice`	An enum has an unexpected value.
`UnusableTripNotice`	Trips must have more than one stop to be usable.
`UnusedShapeNotice`	Shape is not used in GTFS file `trips.txt`.
`UnusedTripNotice`	Trip is not be used in `stop_times.txt`

Table of info

Name	Description
`UnknownColumnNotice`	A column name is unknown.
`UnknownFileNotice`	A file is unknown.

Table of system errors

Name	Description
`IOError`	Error in IO operation.
`RuntimeExceptionInLoaderError`	RuntimeException while loading GTFS dataset in memory.
`RuntimeExceptionInValidatorError`	RuntimeException while validating GTFS archive.
`ThreadExecutionError`	ExecutionException during multithreaded validation
`ThreadInterruptedError`	InterruptedExecution during multithreaded validation
`URISyntaxError`	A string could not be parsed as a URI reference.

Notices

Errors

BlockTripsWithOverlappingStopTimesNotice

Trips with the same block id have overlapping stop times.

References:

Original Python validator implementation

DecreasingOrEqualShapeDistanceNotice

When sorted by shape.shape_pt_sequence, two consecutive shape points should have increasing values for shape_dist_traveled. If the values are equal, this is considered as an error.

References:

shapes.txt specification

DecreasingOrEqualStopTimeDistanceNotice

When sorted by stop_times.stop_pt_sequence, two consecutive stop times in a trip should have increasing distance. If the values are equal, this is considered as an error.

References:

stops.txt specification

DuplicatedColumnNotice

The input file CSV header has the same column name repeated.

References:

Original Python validator implementation

DuplicateFareRuleZoneIdFieldsNotice

The combination of fare_rules.route_id, fare_rules.origin_id, fare_rules.contains_id and fare_rules.destination_id fields should be unique in GTFS file fare_rules.txt.

References:

Original Python validator implementation

DuplicateKeyNotice

The values of the given key and rows are duplicates.

References:

Original Python validator implementation

EmptyFileNotice

Empty csv file found in the archive: file does not have any headers, or is a required file and does not have any data. The GTFS specification requires the first line of each file to contain field names and required files must have data.

References:

GTFS files requirements

ForeignKeyViolationNotice

The values of the given key and rows of one table cannot be found a values of the given key in another table.

References:

Original Python validator implementation

InconsistentAgencyTimezoneNotice

Agencies from GTFS agency.txt have been found to have different timezones.

References:

GTFS agency.txt specification

InvalidColorNotice

Value of field with type color is not valid. A color must be encoded as a six-digit hexadecimal number. The leading "#" is not included.

References:

Field Types Description

InvalidCurrencyNotice

Value of field with type currency is not valid. Currency code must follow ISO 4217

References:

Field Types Description

InvalidDateNotice

Value of field with type date is not valid. Dates must have the YYYYMMDD format.

References:

Field Types Description

InvalidEmailNotice

Value of field with type email is not valid. Definitions for valid emails are quite vague. We perform strict validation in the upstream using the Apache Commons EmailValidator.

References:

InvalidFloatNotice

Value of field with type float is not valid.

References:

Field Types Description

InvalidIntegerNotice

Value of field with type integer is not valid.

References:

Field Types Description

InvalidLanguageCodeNotice

Value of field with type language is not valid. Language codes must follow IETF BCP 47.

References:

Field Types Description

InvalidPhoneNumberNotice

Value of field with type phone number is not valid.

References:

Field Types Description

InvalidRowLengthNotice

A row in the input file has a different number of values than specified by the CSV header.

References:

Original Python validator implementation

InvalidTimeNotice

Value of field with type time is not valid. Time must be in the H:MM:SS, HH:MM:SS or HHH:MM:SS format.

References:

Field Types Description

InvalidTimezoneNotice

Value of field with type timezone is not valid.Timezones are defined at www.iana.org. Timezone names never contain the space character but may contain an underscore. Refer to Wikipedia for a list of valid values.

References:

Field Types Description

InvalidUrlNotice

Value of field with type url is not valid. Definitions for valid URLs are quite vague. We perform strict validation in the upstream using the Apache Commons UrlValidator.

References:

LeadingOrTrailingWhitespacesNotice

The value in CSV file has leading or trailing whitespaces.

References:

GTFS file requirements

LocationWithoutParentStationNotice

A location that must have parent_station field does not have it. The following location types must have parent_station: entrance, generic node, boarding_area.

References:

stops.txt specification

MissingCalendarAndCalendarDateFilesNotice

Both files calendar_dates.txt and calendar.txt are missing from the GTFS archive. At least one of the files must be provided.

References:

MissingRequiredColumnNotice

A required column is missing in the input file.

References:

GTFS terms definition

MissingRequiredFieldNotice

The given field has no value in some input row, even though values are required.

References:

GTFS terms definition

MissingRequiredFileNotice

A required file is missing.

References:

GTFS terms definition

MissingTripEdgeNotice

First and last stop of a trip must define both arrival_time and departure_time fields.

References:

stop_times.txt specification

NewLineInValueNotice

A value in CSV file has a new line or carriage return.

References:

GTFS file requirements

NumberOutOfRangeNotice

The values in the given column of the input rows are out of range.

References:

GTFS file requirements

References:

OverlappingFrequencyNotice

Trip frequencies must not overlap in time

References:

frequencies.txt specification

RouteBothShortAndLongNameMissingNotice

Both short_name and long_name are missing for a route.

References:

routes.txt specification

SameNameAndDescriptionForRouteNotice

The GTFS spec defines routes.txt route_description as:

Description of a route that provides useful, quality information. Do not simply duplicate the name of the route.

See the GTFS and GTFS Best Practices links below for more examples of how to populate the route_short_name, route_long_name, and route_description fields.

References:

routes.txt specification routes.txt Best Practices

StartAndEndRangeEqualNotice

Date or time fields have been found equal.

References:

Original Python validator implementation

StartAndEndRangeOutOfOrderNotice

Date or time fields have been found out of order.

References:

Original Python validator implementation

StationWithParentStationNotice

Field parent_station must be empty when location_type is 2.

References:

stop_times.txt

StopTimeWithArrivalBeforePreviousDepartureTimeNotice

For a given trip_id, the arrival_time of (n+1)-th stoptime in sequence must not precede the departure_time of n-th stoptime in sequence.

References:

Original Python validator implementation

StopTimeWithDepartureBeforeArrivalTimeNotice

The departure_time must not precede the arrival_time in stop_times.txt if both are given.

References:

Original Python validator implementation

StopTimeWithOnlyArrivalOrDepartureTimeNotice

Missing stop_time.arrival_time or stop_time.departure_time

References:

stop_times.txt specification

WrongParentLocationTypeNotice

Value of field location_type of parent found in field parent_station is invalid.

According to spec

Stop/platform can only have Station as parent
Station can NOT have a parent
Entrance/exit or generic node can only have Station as parent
Boarding Area can only have Platform as parent

Any other combination raise this error.

References:

stops.txt specification

Warnings

AttributionWithoutRoleNotice

At least one of the fields is_producer, is_operator, or is_authority should be set to 1.

References:

attributions.txt specification

DuplicateRouteNameNotice

All routes should have different routes.route_long_name - if two routes.route_long_name are the same, and the two routes belong to the same agency, a notice is generated.

Note that there may be valid cases where routes may have the same routes.route_long_name and this notice can be ignored. For example, routes may have the same routes.route_long_name if they serve difference areas. However, they must not be different trips of the same route or different directions of the same route - these cases should always have unique routes.route_long_name.

All routes should have different routes.route_short_name - if two routes.route_short_name are the same, and the two routes belong to the same agency, a notice is generated.

Note that there may be valid cases where routes may have the same routes.route_short_name and this notice can be ignored. For example, routes may have the same routes.route_short_name if they serve difference areas. However, they must not be different trips of the same route or different directions of the same route - these cases should always have unique routes.route_short_name.

The same combination of route_short_name and route_long_name should not be used for more than one route.

References:

routes.txt specification

EmptyColumnNameNotice

A column name has not been provided. Such columns are skipped by the validator.

References:

GTFS file requirements

EmptyRowNotice

A row in the input file has only spaces.

References:

GTFS file requirements

FeedExpirationDateNotice

At any time, the published GTFS dataset should be valid for at least the next 7 days, and ideally for as long as the operator is confident that the schedule will continue to be operated. If possible, the GTFS dataset should cover at least the next 30 days of service.

References:

General Publishing & General Practices

FeedInfoLangAndAgencyLangMismatchNotice

Files agency.txt and feed_info.txt must define matching agency.agency_lang and feed_info.feed_lang. The default language may be multilingual for datasets with the original text in multiple languages. In such cases, the feed_lang field should contain the language code mul defined by the norm ISO 639-2.

If feed_lang is not mul and does not match with agency_lang, that's an error
If there is more than one agency_lang and feed_lang isn't mul, that's an error
If feed_lang is mul and there isn't more than one agency_lang, that's an error

References:

InconsistentAgencyLangNotice

Agencies from GTFS agency.txt have been found to have different languages.

References:

Original Python validator implementation

MissingFeedInfoDateNotice

Even though feed_info.start_date and feed_info.end_date are optional, if one field is provided the second one should also be provided.

References:

feed_info.txt Best practices

MoreThanOneEntityNotice

The file is expected to have a single entity but has more (e.g., "feed_info.txt").

References:

GTFS field definition

NonAsciiOrNonPrintableCharNotice

A value of filed with type id contains non ASCII or non printable characters. This is not recommended.

References:

Original Python validator implementation

PlatformWithoutParentStationNotice

A platform has no parent_station field set.

References:

stops.txt specification

RouteColorContrastNotice

A route's color and route_text_color should be contrasting.

References:

RouteShortAndLongNameEqualNotice

Short and long name are equal for a route.

References:

routes.txt specification

RouteShortNameTooLongNotice

Short name of a route is too long (more than 12 characters).

References:

routes.txt Best Practices

StartAndEndTimeEqualNotice

Start and end times are equal in GTFS file frequencies.txt. The GTFS spec is currently unclear how this case should be handled (e.g., is it a trip that circulates once?). It is recommended to use a trip not defined via frequencies.txt for this case.

References:

Original Python validator implementation

StopTimeTimepointWithoutTimesNotice

Any record with stop_times.timepoint set to 1 should define a value for stop_times.arrival_time and stop_times.departure_time fields.

References:

GTFS stop_times.txt specification

StopTooFarFromTripShapeNotice

Per GTFS Best Practices, route alignments (in shapes.txt) should be within 100 meters of stop locations which a trip serves.

References:

GTFS Best Practices shapes.txt

TooFastTravelNotice

As implemented in the original Google Python GTFS validator, the calculated speed between stops should not be greater than 150 km/h (42 m/s SI or 93 mph).