Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pickup and drop off window validator #1935

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

feat: pickup and drop off window validator #1935

wants to merge 5 commits into from

Conversation

cka-y
Copy link
Contributor

@cka-y cka-y commented Dec 24, 2024

Summary:

  • Improved the dynamic handling of validation notice columns in the HTML report to ensure consistent ordering and proper mapping of fields. This issue did not affect the JSON report, which was already functioning correctly.
  • Added new validation rules and corresponding notices:
    • forbidden_arrival_or_departure_time: Triggers when both arrival/departure times and pickup/drop-off windows are defined, which is not allowed.
    • missing_pickup_or_drop_off_window: Identifies cases where either the start or end pickup/drop-off window is missing but the other is present.
    • invalid_pickup_drop_off_window: Flags scenarios where the end pickup/drop-off window occurs before or at the same time as the start window.

Expected behavior:

  • forbidden_arrival_or_departure_time: This example showcases the dynamic display of columns (the departure_time column is missing from display as it is not defined in the tested feed)
    image

  • missing_pickup_or_drop_off_window:
    image

  • invalid_pickup_drop_off_window:
    image

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with gradle test to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@MobilityData MobilityData deleted a comment from github-actions bot Dec 24, 2024
@MobilityData MobilityData deleted a comment from github-actions bot Dec 24, 2024
@cka-y cka-y changed the title Feat/1816 feat: pickup and drop off window validator Dec 24, 2024
@MobilityData MobilityData deleted a comment from github-actions bot Dec 24, 2024
Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit 587efd4
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (2 out of 1788 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-colorado-tsc-transit-gtfs-2052 invalid_pickup_drop_off_window
us-colorado-estes-transit-gtfs-2047 missing_pickup_or_drop_off_window
Dropped Errors (0 out of 1788 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (0 out of 1788 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1788 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1788 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.94 4.01 ⬆️+0.07
Median -- 1.39 1.45 ⬆️+0.05
Standard Deviation -- 11.02 11.09 ⬆️+0.06
Minimum in References Reports us-california-flex-v2-developer-test-feed-3-gtfs-1819 0.53 0.54 ⬆️+0.01
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 286.81 289.48 ⬆️+2.67
Minimum in Latest Reports us-massachusetts-massachusetts-area-express-max-gtfs-431 0.56 0.53 ⬇️-0.03
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 286.81 289.48 ⬆️+2.67
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 468.97 MiB 483.86 MiB ⬆️+14.89 MiB
Median -- 244.49 MiB 246.50 MiB ⬆️+2.01 MiB
Standard Deviation -- 846.45 MiB 881.96 MiB ⬆️+35.51 MiB
Minimum in References Reports us-oregon-hut-airport-shuttle-gtfs-635 34.48 MiB 34.52 MiB ⬆️+42.16 KiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 9.82 GiB 10.22 GiB ⬆️+405.17 MiB
Minimum in Latest Reports us-michigan-detroit-people-mover-gtfs-417 34.50 MiB 34.50 MiB ⬇️0 bytes
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 9.82 GiB 10.22 GiB ⬆️+405.17 MiB

Copy link
Member

@davidgamez davidgamez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

}

@Override
public boolean shouldCallValidate(ColumnInspector header) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants