Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable Local Zones in Acceptance Testing Configurations #12412

Closed
bflad opened this issue Mar 16, 2020 · 1 comment · Fixed by #12517
Closed

Disable Local Zones in Acceptance Testing Configurations #12412

bflad opened this issue Mar 16, 2020 · 1 comment · Fixed by #12517
Assignees
Labels
provider Pertains to the provider itself, rather than any interaction with AWS. service/ec2 Issues and PRs that pertain to the ec2 service. technical-debt Addresses areas of the codebase that need refactoring or redesign. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.

Comments

@bflad
Copy link
Contributor

bflad commented Mar 16, 2020

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Support for Local Zones in the Terraform AWS Provider is in progress, however the provider-wide daily acceptance testing found an interesting caveat in us-west-2 (the default testing region) after us-west-2-lax-1 was enabled. The aws_availability_zones data source (correctly) will automatically include this new result, but it is first in the resultant list. Since Local Zones have limited functionality and mixing Local Zones and Availability Zones seems to be unsupported in many cases (for configurations that require multi-AZ testing), a large percentage of new errors occurred. Local Zones can only be disabled via AWS Support, so while we can submit this case, it seems it would be better to just bulk update the affected test configurations to prevent this issue going forward.

Affected Resource(s)

Daily acceptance testing: Tests failed: 623 (507 new), passed: 2718, ignored: 64

Potential Terraform Configuration

data "aws_availability_zones" "available" {
  state = "available"

  # New configuration to exclude Local Zones
  filter {
    name  = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

References

@bflad bflad added tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. technical-debt Addresses areas of the codebase that need refactoring or redesign. provider Pertains to the provider itself, rather than any interaction with AWS. labels Mar 16, 2020
@bflad bflad self-assigned this Mar 16, 2020
@ghost ghost added the service/ec2 Issues and PRs that pertain to the ec2 service. label Mar 16, 2020
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Mar 16, 2020
@bflad bflad removed the needs-triage Waiting for first response or review from a maintainer. label Mar 16, 2020
bflad added a commit that referenced this issue Mar 24, 2020
…st configurations to exclude Local Zones

Reference: #12412

This is the least risky update for all testing to exclude Local Zones from the data source, since us-west-2-lax-1 generally is missing functionality and also to prevent Availability Zone and Local Zone mixing when multiple Availability Zones are required, which is not allowed in many services.

In the future, this type of configuration churn can be reduced with one of the following solutions:

- Removing aws_availability_zones data source usage where it has now been superceded by the creation of the aws_ec2_instance_type_offering data source
- Standardizing base configurations per data source/resource
- Standardizing shared configurations into helper test configurations and composing them

Output from acceptance testing (as a smoke test):

```
--- PASS: TestAccAWSLB_ALB_basic (197.43s)
```
bflad added a commit that referenced this issue Mar 25, 2020
…st configurations to exclude Local Zones (#12517)

* tests/provider: Bulk update aws_availability_zones data sources in test configurations to exclude Local Zones

Reference: #12412

This is the least risky update for all testing to exclude Local Zones from the data source, since us-west-2-lax-1 generally is missing functionality and also to prevent Availability Zone and Local Zone mixing when multiple Availability Zones are required, which is not allowed in many services.

In the future, this type of configuration churn can be reduced with one of the following solutions:

- Removing aws_availability_zones data source usage where it has now been superceded by the creation of the aws_ec2_instance_type_offering data source
- Standardizing base configurations per data source/resource
- Standardizing shared configurations into helper test configurations and composing them

Output from acceptance testing (as a smoke test):

```
--- PASS: TestAccAWSLB_ALB_basic (197.43s)
```

* docs/provider: Include Local Zones filtering in Contributing Guide

Reference: #12517 (comment)

* data-source/aws_vpc_endpoint_service: Keep non-filtered aws_availability_zones data source configuration

Since the expected return value of availability_zones applies to all Availability Zones and Local Zones:

```
--- FAIL: TestAccDataSourceAwsVpcEndpointService_gateway (6.13s)
    testing.go:654: Step 0 error: Check failed: Check 3/10 error: data.aws_vpc_endpoint_service.test: Attribute 'availability_zones.#' expected "4", got "5"
```

Output from acceptance testing:

```
--- PASS: TestAccDataSourceAwsVpcEndpointService_gateway (15.03s)
```

* tests/data-source/aws_availability_zone: Exclude Local Zones from aws_availability_zones results in test configurations

Previously:

```
--- FAIL: TestAccDataSourceAwsAvailabilityZone_Filter (9.96s)
    testing.go:654: Step 0 error: Check failed: Check 1/7 error: data.aws_availability_zone.test: Attribute 'group_name' expected "us-west-2", got "us-west-2-lax-1"
```

Output from acceptance testing:

```
--- PASS: TestAccDataSourceAwsAvailabilityZone_Filter (16.70s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_AllAvailabilityZones (16.97s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_Name (17.17s)
--- PASS: TestAccDataSourceAwsAvailabilityZone_ZoneId (17.28s)
```
@ghost
Copy link

ghost commented Apr 25, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
provider Pertains to the provider itself, rather than any interaction with AWS. service/ec2 Issues and PRs that pertain to the ec2 service. technical-debt Addresses areas of the codebase that need refactoring or redesign. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
1 participant