Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support metric collections for autoscaling group #6712

Closed
wants to merge 3 commits into from

Conversation

fogfish
Copy link
Contributor

@fogfish fogfish commented Mar 13, 2020

fixes #6453

@mergify
Copy link
Contributor

mergify bot commented Mar 13, 2020

Title does not follow the guidelines of Conventional Commits. Please adjust title before merge.

@fogfish fogfish changed the title Support metric collections for autoscaling group fix: support metric collections for autoscaling group Mar 13, 2020
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: ea76912
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 4afadce
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 87f7f43
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@NetaNir NetaNir self-requested a review March 18, 2020 22:13
Copy link
Contributor

@NetaNir NetaNir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HI @fogfish!

Thanks for submitting the PR!
I started reviewing it and I think there might be an opportunity to make this API neat:)
Given metricsCollection is a list of metrics Collection object:

{
      granularity: '1Minute'
      metrics: [] || undefined
}

Since granularity is a required property with a single allowed value - 1Minute, it make sense for the CDK to make it optional and have a default value of 1Minute. But, given that metrics is also optional as with a defaults to allMetrics this will result is a somewhat weird API, to collect all metrics in the default granularity the user code will be:

metricsCollections: [{}]

To make the API cleaner we can leverage the "union like class" pattern we use in the CDK. The union like pattern allows the initialization of a type using different parameters set. Here is a sketch of the code:

export interface MetricsCollectionOptions {
    readonly granularity?: Granularity;
    readonly metrics: MetricGroup[];
}

export interface MetricsCollectionProps {
  readonly granularity: Granularity;
  readonly metrics: MetricGroup[];
}

export class MetricsCollections {

  /**
   * Enable the monitoring of all metrics of the Auto Scaling group
   * @param granularity The frequency at which Amazon EC2 Auto Scaling sends aggregated data to CloudWatch (default 1Minute)
   */
  public static allMetrics(granularity?: Granularity): MetricsCollections {
    return new MetricsCollections({
      granularity: granularity ?? Granularity.ONE_MINUTE,
      metrics: []
    });
  }

/**
 * 
 * @param metricsCollections 
 */
  public static groups(...metricsCollections: MetricsCollectionOptions[]): MetricsCollections {
    return new MetricsCollections(...metricsCollections.map(mc => ({
      granularity: mc.granularity ?? Granularity.ONE_MINUTE,
      metrics: mc.metrics
    })));
  }

  public readonly collections: MetricsCollectionProps[];

  private constructor(...metricsCollections: MetricsCollectionProps[]) {
    this.collections = metricsCollections;
  }

}

And the API will look like this:

new autoscaling.AutoScalingGroup(stack, 'MyFleet', {
  metricCollections: MetricsCollections.allMetrics(),
// other properties
}

To collect all metrics.

or:

new autoscaling.AutoScalingGroup(stack, 'MyFleet', {
  metricCollections: MetricsCollections.groups([
    {
       granularity: Granularity.ONE_MINUTE,
        metrics: [MetricsGroup. GROUP_IN_SERVICE_INSTANCES]
     }
   ]),
// other properties
}

To collect different metricsGroups in different granularity.

I know it's a lot but I would be happy to do another round of review if you want to pick it up.
I'm not attached to any of the names so feel free to use other names.
Good examples for the union like class in the CDK are Schedule in aws-application-autoscaling and Code in aws-lambda.

Thanks again!

Comment on lines +50 to +75
export enum GroupMetric {
/** GroupMinSize */
MIN_SIZE = "GroupMinSize",

/** GroupMaxSize */
MAX_SIZE = "GroupMaxSize",

/** GroupDesiredCapacity */
DESIRED_CAPACITY = "GroupDesiredCapacity",

/** GroupInServiceInstances */
IN_SERVICE_INSTANCES = "GroupInServiceInstances",

/** GroupPendingInstances */
PENDING_INSTANCES = "GroupPendingInstances",

/** GroupStandbyInstances */
STANDBY_INSTANCES = "GroupStandbyInstances",

/** GroupTerminatingInstances */
TERMINATING_INSTANCES = "GroupTerminatingInstances",

/** GroupTotalInstances */
TOTAL_INSTANCES = "GroupTotalInstances",
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the docs doesn't add any information there is no need to add them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to remove these comments but the build system gonna give me a error that documentation is required for exported types.

* Auto Scaling group metrics
*/
export enum GroupMetric {
/** GroupMinSize */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MetricGroup

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Change to MetricGroup

* https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-monitoring.html
*/
export interface AutoScalingGroupMetric {
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can name this MetricsCollection so it will match the actual property.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Change to MetricsCollection

Comment on lines +24 to +29
/**
* Describes the group metrics that an Amazon EC2 Auto Scaling group sends to Amazon CloudWatch.
* These metrics describe the group rather than any of its instances.
* For more information, see Monitoring Your Auto Scaling Groups and Instances Using Amazon CloudWatch in the Amazon EC2 Auto Scaling User Guide.
* https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-monitoring.html
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this describe MetricCollection this should refer to the MetricCollection doc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Update the link in the doc

* The frequency at which Amazon EC2 Auto Scaling sends aggregated data to CloudWatch.
*/
readonly granularity: MetricGranularity,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be optional and have a default value of 1Minute which is the only allowed value.

* The frequency at which Amazon EC2 Auto Scaling sends aggregated data to CloudWatch.
*/
export type MetricGranularity = "1Minute";

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be an Enum

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You build system gives an error if I create enum with one entry. This is exactly how I've implemented at first time.

Comment on lines +36 to +38
/**
* The list of Auto Scaling group metrics to collect. If you specify Granularity and don't specify any metrics, all metrics are enabled.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/**
* The list of Auto Scaling group metrics to collect. If you specify Granularity and don't specify any metrics, all metrics are enabled.
*/
/**
* The list of Auto Scaling group metrics to collect.
* @deafult - all metrics are enabled
*/

Use the @default - notation here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • fix typo

*
* @default - metrics collection is disabled.
*/
readonly metricsCollection?: AutoScalingGroupMetric[]
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to metricsCollections

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/**
* The list of Auto Scaling group metrics to collect. If you specify Granularity and don't specify any metrics, all metrics are enabled.
*/
readonly metrics: GroupMetric[],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this optional

@@ -514,6 +573,7 @@ export class AutoScalingGroup extends AutoScalingGroupBase implements
vpcZoneIdentifier: subnetIds,
healthCheckType: props.healthCheck && props.healthCheck.type,
healthCheckGracePeriod: props.healthCheck && props.healthCheck.gracePeriod && props.healthCheck.gracePeriod.toSeconds(),
metricsCollection: props.metricsCollection || undefined,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no need to add the || undefined

@NetaNir NetaNir added the needs-discussion This issue/PR requires more discussion with community. label Mar 19, 2020
@fogfish
Copy link
Contributor Author

fogfish commented Mar 20, 2020

Thank you very much for your suggestion about the union pattern. I think it is helpful in may use-cases. Unfortunately, I've confused about its usability from developers perspective in this particular use-case. Let's step into developer shoe:

As a developer I want to enable ASG monitoring so that ...

As a developer I want to enable monitoring of specific metrics so that ...

It became more friendly for developer to use boolean flag just to enable monitoring. Therefore I would tune ASG properties to accept Sum types

class ASGProps {
  readonly enableMonitoring: boolean | MetricsCollection[]
}


new autoscaling.AutoScalingGroup(stack, 'MyFleet', {
   enableMonitoring: true
}

new autoscaling.AutoScalingGroup(stack, 'MyFleet', {
  enableMonitoring: [
     {
       granularity: Granularity.ONE_MINUTE,
       metrics: [MetricsGroup. GROUP_IN_SERVICE_INSTANCES]
     },
  ]
}

@fogfish
Copy link
Contributor Author

fogfish commented Mar 25, 2020

@NetaNir Any thoughts? I would not make any change to the PR until we agree about API.

RomainMuller added a commit to aws/jsii that referenced this pull request Mar 26, 2020
TypeScript represents enums similar to union types, and single-valued
enums are "simplified" to the sole member (the TypeChecker fiercely
refuses to give a handle to the actual `enum` type). This resulted in
`jsii` incorrectly tagging the type at usage sites.

This commit adds the necessary infrastructure to detect single-valued
enums and tweak the FQN generation to obtain the correct result.

A new test was added to validate this whole endeavor works correctly
even when the single-valued enum is within a submodule (which adds even
more complexity to the mix).

References: aws/aws-cdk#6712 aws/aws-cdk#6948
@aws aws deleted a comment from mergify bot Mar 26, 2020
mergify bot pushed a commit to aws/jsii that referenced this pull request Mar 27, 2020
TypeScript represents enums similar to union types, and single-valued
enums are "simplified" to the sole member (the TypeChecker fiercely
refuses to give a handle to the actual `enum` type). This resulted in
`jsii` incorrectly tagging the type at usage sites.

This commit adds the necessary infrastructure to detect single-valued
enums and tweak the FQN generation to obtain the correct result.

A new test was added to validate this whole endeavor works correctly
even when the single-valued enum is within a submodule (which adds even
more complexity to the mix).

References: aws/aws-cdk#6712 aws/aws-cdk#6948
@NetaNir
Copy link
Contributor

NetaNir commented Mar 31, 2020

Hi @fogfish!

The single enum bug was fixed!

As for the API, the experience in Java, C# and typically other statically typed languages without type unions will be inadequate, which is why we try to avoid it in the CDK (we currently can't prevent it in jsii since it will be a breaking change) and use the union like pattern.

@huangzbaws
Copy link

Is there any update for this issue? My project is still waiting for this feature to be closed. Thanks!

@NetaNir
Copy link
Contributor

NetaNir commented May 11, 2020

Closing this PR in favor of #7432

@NetaNir NetaNir closed this May 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-discussion This issue/PR requires more discussion with community.
Projects
None yet
4 participants