Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests/provider: Bulk update aws_availability_zones data sources in test configurations to exclude Local Zones #12517

Merged
merged 4 commits into from
Mar 25, 2020
Merged

tests/provider: Bulk update aws_availability_zones data sources in test configurations to exclude Local Zones #12517

merged 4 commits into from
Mar 25, 2020

Conversation

bflad
Copy link
Contributor

@bflad bflad commented Mar 24, 2020 •
edited
Loading

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #12412

Release note for CHANGELOG:

NONE

This is the least risky update for all testing to exclude Local Zones from the data source, since us-west-2-lax-1 generally is missing functionality and also to prevent Availability Zone and Local Zone mixing when multiple Availability Zones are required, which is not allowed in many services.

In the future, this type of configuration churn can be reduced with one (or all) of the following solutions:

  • Removing aws_availability_zones data source usage where it has now been superseded by the creation of the aws_ec2_instance_type_offering data source
  • Standardizing base configurations per data source/resource
  • Standardizing shared configurations into helper test configurations and composing them (requires design work)

Output from acceptance testing (as a smoke test):

--- PASS: TestAccAWSLB_ALB_basic (197.43s)

@bflad
tests/provider: Bulk update aws_availability_zones data sources in te…
2186150 
…st configurations to exclude Local Zones

Reference: #12412

This is the least risky update for all testing to exclude Local Zones from the data source, since us-west-2-lax-1 generally is missing functionality and also to prevent Availability Zone and Local Zone mixing when multiple Availability Zones are required, which is not allowed in many services.

In the future, this type of configuration churn can be reduced with one of the following solutions:

- Removing aws_availability_zones data source usage where it has now been superceded by the creation of the aws_ec2_instance_type_offering data source
- Standardizing base configurations per data source/resource
- Standardizing shared configurations into helper test configurations and composing them

Output from acceptance testing (as a smoke test):

```
--- PASS: TestAccAWSLB_ALB_basic (197.43s)
```
@bflad bflad requested a review from a team March 24, 2020 23:31
@ghost ghost added size/XXL Managed by automation to categorize the size of a PR. needs-triage Waiting for first response or review from a maintainer. service/apigateway Issues and PRs that pertain to the apigateway service. service/applicationautoscaling service/autoscaling Issues and PRs that pertain to the autoscaling service. service/ec2 Issues and PRs that pertain to the ec2 service. service/elasticache Issues and PRs that pertain to the elasticache service. service/elasticsearch Issues and PRs that pertain to the elasticsearch service. service/elb Issues and PRs that pertain to the elb service. service/elbv2 Issues and PRs that pertain to the elbv2 service. service/mq Issues and PRs that pertain to the mq service. service/rds Issues and PRs that pertain to the rds service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Mar 24, 2020
@bflad bflad removed the needs-triage Waiting for first response or review from a maintainer. label Mar 24, 2020
@ewbankkit
Copy link
Contributor

Also should update the Uses aws_availability_zones Data Source checklist item in CONTRIBUTING.md.

@bflad
Copy link
Contributor Author

bflad commented Mar 25, 2020

@ewbankkit done

bflad added 2 commits March 25, 2020 14:46
@bflad
data-source/aws_vpc_endpoint_service: Keep non-filtered aws_availabil…
cbccf93 
…ity_zones data source configuration

Since the expected return value of availability_zones applies to all Availability Zones and Local Zones:

```
--- FAIL: TestAccDataSourceAwsVpcEndpointService_gateway (6.13s)
    testing.go:654: Step 0 error: Check failed: Check 3/10 error: data.aws_vpc_endpoint_service.test: Attribute 'availability_zones.#' expected "4", got "5"
```

Output from acceptance testing:

```
--- PASS: TestAccDataSourceAwsVpcEndpointService_gateway (15.03s)
```
@bflad
tests/data-source/aws_availability_zone: Exclude Local Zones from aws…
084558f 
…_availability_zones results in test configurations

Previously:

```
--- FAIL: TestAccDataSourceAwsAvailabilityZone_Filter (9.96s)
    testing.go:654: Step 0 error: Check failed: Check 1/7 error: data.aws_availability_zone.test: Attribute 'group_name' expected "us-west-2", got "us-west-2-lax-1"
```

Output from acceptance testing:

```
--- PASS: TestAccDataSourceAwsAvailabilityZone_Filter (16.70s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_AllAvailabilityZones (16.97s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_Name (17.17s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_ZoneId (17.28s)
```
gdavison
gdavison approved these changes Mar 25, 2020
View reviewed changes
Copy link
Contributor

@gdavison gdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

--- PASS: TestAccDataSourceAwsAvailabilityZone_Filter (10.92s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_ZoneId (11.58s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_AllAvailabilityZones (11.77s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_Name (12.16s)
--- PASS: TestAccAWSEc2InstanceTypeOfferingDataSource_Filter (12.44s)
--- PASS: TestAccAWSEc2InstanceTypeOfferingDataSource_PreferredInstanceTypes (13.86s)
--- PASS: TestAccAWSEc2InstanceTypeOfferingDataSource_LocationType (14.65s)
--- PASS: TestAccAWSEc2InstanceTypeOfferingsDataSource_Filter (8.17s)
--- PASS: TestAccAWSEc2InstanceTypeOfferingsDataSource_LocationType (8.41s)
--- PASS: TestAccDataSourceAwsEip_Filter (9.29s)
--- PASS: TestAccDataSourceAwsEip_Id (9.02s)
--- PASS: TestAccAWSEbsVolumeDataSource_basic (23.07s)
--- PASS: TestAccAWSEbsVolumeDataSource_multipleFilters (23.32s)
--- PASS: TestAccDataSourceAwsEip_PublicIP_VPC (9.13s)
--- PASS: TestAccDataSourceAwsEip_Tags (9.10s)
--- PASS: TestAccAWSEbsSnapshotDataSource_Filter (38.15s)
--- PASS: TestAccAWSEbsSnapshotDataSource_basic (38.32s)
--- PASS: TestAccDataSourceAwsEip_PublicIP_EC2Classic (24.78s)
--- PASS: TestAccDataSourceAwsEip_NetworkInterface (22.48s)
--- PASS: TestAccAwsAutoScalingGroupDataSource_basic (50.79s)
--- PASS: TestAccDataSourceAWSELB_basic (15.35s)
--- PASS: TestAccAWSEbsSnapshotDataSource_MostRecent (54.58s)
--- PASS: TestAccAWSAutoscalingGroups_basic (87.90s)
--- PASS: TestAccDataSourceAwsEip_Instance (107.34s)
--- FAIL: TestAccAWSDbClusterSnapshotDataSource_MostRecent (169.48s)
--- PASS: TestAccAWSDbClusterSnapshotDataSource_DbClusterSnapshotIdentifier (172.07s)
--- PASS: TestAccAWSDbClusterSnapshotDataSource_DbClusterIdentifier (172.33s)
--- PASS: TestAccDataSourceAwsNetworkInterface_basic (13.48s)
--- PASS: TestAccDataSourceAwsNetworkInterface_filters (13.28s)
--- PASS: TestAccDataSourceAwsSubnet_basic (11.30s)
--- PASS: TestAccDataSourceAWSLB_basic (152.19s)
--- PASS: TestAccDataSourceAwsSubnet_ipv6ByIpv6Filter (18.48s)
--- PASS: TestAccDataSourceAWSLBListener_BackwardsCompatibility (182.64s)
--- PASS: TestAccDataSourceAWSLBTargetGroupBackwardsCompatibility (169.95s)
--- PASS: TestAccDataSourceAwsVpcEndpointService_gateway (8.48s)
--- PASS: TestAccDataSourceAWSLBListener_https (181.54s)
--- PASS: TestAccDataSourceAwsSubnet_ipv6ByIpv6CidrBlock (19.15s)
--- FAIL: TestAccDataSourceAwsVpcEndpointService_interface (5.78s)
--- PASS: TestAccDataSourceAWSLBListener_basic (201.32s)
--- PASS: TestAccDataSourceAWSALBTargetGroup_basic (190.10s)
--- PASS: TestAccDataSourceAwsVpcEndpoint_gatewayBasic (22.18s)
--- PASS: TestAccDataSourceAwsVpcEndpoint_byId (21.40s)
--- PASS: TestAccDataSourceAwsVpcEndpoint_byFilter (22.42s)
--- PASS: TestAccDataSourceAwsVpcEndpoint_byTags (22.51s)
--- PASS: TestAccDataSourceAwsVpcEndpoint_gatewayWithRouteTableAndTags (21.42s)
--- PASS: TestAccDataSourceCloudHsmV2Cluster_basic (275.58s)
--- PASS: TestAccDataSourceAWSLBBackwardsCompatibility (190.37s)
--- PASS: TestAccAWSAMI_basic (45.65s)
--- PASS: TestAccAWSAMI_disappears (40.90s)
--- PASS: TestAccAWSAMI_tags (57.54s)
--- PASS: TestAccAWSAPIGatewayIntegration_basic (29.10s)
--- PASS: TestAccAWSAMI_snapshotSize (45.67s)
--- PASS: TestAccAWSRouteDataSource_basic (196.63s)
--- PASS: TestAccAWSAPIGatewayIntegration_contentHandling (44.53s)
--- PASS: TestAccDataSourceAwsVpcEndpoint_interface (143.16s)
--- PASS: TestAccAWSAPIGatewayIntegration_cache_key_parameters (62.72s)
--- PASS: TestAccAWSAPIGatewayRestApi_disappears (57.21s)
--- PASS: TestAccAWSAPIGatewayRestApi_basic (99.54s)
--- PASS: TestAccDataSourceAwsVpcEndpointService_custom (257.03s)
--- PASS: TestAccDataSourceAwsDirectoryServiceDirectory_SimpleAD (538.90s)
--- PASS: TestAccAWSAPIGatewayRestApi_api_key_source (60.57s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration (141.77s)
--- PASS: TestAccAWSAPIGatewayRestApi_policy (59.60s)
--- FAIL: TestAccAWSRouteDataSource_TransitGatewayID (416.73s)
--- PASS: TestAccAWSAMICopy_basic (358.94s)
--- PASS: TestAccAWSAMICopy_Description (363.99s)
--- PASS: TestAccAWSAMICopy_EnaSupport (367.85s)
--- PASS: TestAccAWSAMICopy_tags (378.27s)
--- PASS: TestAccAWSEc2TransitGatewayVpcAttachmentDataSource_Filter (669.99s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_Private (242.83s)
--- PASS: TestAccAWSEc2TransitGatewayVpcAttachmentDataSource_ID (690.24s)
--- PASS: TestAccAWSAPIGatewayRestApi_openapi (222.61s)
--- PASS: TestAccAWSAPIGatewayRestApi_tags (397.64s)
--- PASS: TestAccAWSDataElasticsearchDomain_basic (750.53s)
--- PASS: TestAccDataSourceAwsElasticacheReplicationGroup_basic (807.09s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_VPCEndpoint (374.89s)
--- PASS: TestAccDataSourceAwsElasticacheReplicationGroup_ClusterMode (978.13s)
--- PASS: TestAccAWSDataElasticsearchDomain_advanced (1005.52s)
--- PASS: TestAccAWSAPIGatewayIntegration_integrationType (686.17s)
--- PASS: TestAccAWSAPIGatewayVpcLink_basic (660.65s)
--- PASS: TestAccAWSAPIGatewayVpcLink_tags (676.76s)
--- PASS: TestAccDataSourceAWSMqBroker_basic (1275.75s)
--- FAIL: TestAccDataSourceAwsDirectoryServiceDirectory_MicrosoftAD (2210.60s)

The four failed tests seem unrelated to the changes in this PR

  • TestAccAWSDbClusterSnapshotDataSource_MostRecent
  • TestAccDataSourceAwsVpcEndpointService_interface
  • TestAccAWSRouteDataSource_TransitGatewayID
  • TestAccDataSourceAwsDirectoryServiceDirectory_MicrosoftAD

@bflad bflad added this to the v2.55.0 milestone Mar 25, 2020
@bflad bflad merged commit f004cde into master Mar 25, 2020
@bflad bflad deleted the t-disable-local-zones branch March 25, 2020 23:44
@ghost
Copy link

ghost commented Mar 27, 2020

This has been released in version 2.55.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Apr 25, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@gdavison gdavison gdavison approved these changes

Assignees
No one assigned
Labels
service/apigateway Issues and PRs that pertain to the apigateway service. service/autoscaling Issues and PRs that pertain to the autoscaling service. service/ec2 Issues and PRs that pertain to the ec2 service. service/elasticache Issues and PRs that pertain to the elasticache service. service/elasticsearch Issues and PRs that pertain to the elasticsearch service. service/elb Issues and PRs that pertain to the elb service. service/elbv2 Issues and PRs that pertain to the elbv2 service. service/mq Issues and PRs that pertain to the mq service. service/rds Issues and PRs that pertain to the rds service. size/XXL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Milestone
v2.55.0
Development

Successfully merging this pull request may close these issues.

Disable Local Zones in Acceptance Testing Configurations
3 participants
@bflad @ewbankkit @gdavison