[Integration][PagerDuty] - Default Resources Enhancements (#1182)

# Description

What - Added enhancements to the integration default resources as well
as the integration itself.

Why - Rethinking the way we map data from PagerDuty and the need to
improve on the overall user experiences

How - By undertaking the following:
- Improved on the integration default resources for PagerDuty incidents
kind
- Added blueprint and mapping for PagerDuty users
- Added logs to track the HTTP requests made to PagerDuty API
- Added a generic resync event handler to allow the integration to bring
any valid resource from PagerDuty

## Type of change

Please leave one option from the following and delete the rest:

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] New Integration (non-breaking change which adds a new integration)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Non-breaking change (fix of existing functionality that will not
change current behavior)
- [ ] Documentation (added/updated documentation)

<h4> All tests should be run against the port production
environment(using a testing org). </h4>

### Core testing checklist

- [ ] Integration able to create all default resources from scratch
- [ ] Resync finishes successfully
- [ ] Resync able to create entities
- [ ] Resync able to update entities
- [ ] Resync able to detect and delete entities
- [ ] Scheduled resync able to abort existing resync and start a new one
- [ ] Tested with at least 2 integrations from scratch
- [ ] Tested with Kafka and Polling event listeners
- [ ] Tested deletion of entities that don't pass the selector


### Integration testing checklist

- [ ] Integration able to create all default resources from scratch
- [ ] Resync able to create entities
- [ ] Resync able to update entities
- [ ] Resync able to detect and delete entities
- [ ] Resync finishes successfully
- [ ] If new resource kind is added or updated in the integration, add
example raw data, mapping and expected result to the `examples` folder
in the integration directory.
- [ ] If resource kind is updated, run the integration with the example
data and check if the expected result is achieved
- [ ] If new resource kind is added or updated, validate that
live-events for that resource are working as expected
- [ ] Docs PR link [here](#)

### Preflight checklist

- [ ] Handled rate limiting
- [ ] Handled pagination
- [ ] Implemented the code in async
- [ ] Support Multi account

## Screenshots

Include screenshots from your environment showing how the resources of
the integration will look.

## API Documentation

Provide links to the API documentation used for this integration.

integrations/pagerduty/.port/resources/blueprints.json

@@ -1,4 +1,69 @@
 [
+  {
+    "identifier": "pagerdutyUser",
+    "description": "This blueprint represents a PagerDuty user in our software catalog",
+    "title": "PagerDuty User",
+    "icon": "pagerduty",
+    "schema": {
+      "properties": {
+        "role": {
+          "icon": "DefaultProperty",
+          "title": "Role",
+          "type": "string",
+          "enum": [
+            "admin",
+            "user",
+            "observer",
+            "limited_user",
+            "owner",
+            "read_only_user",
+            "restricted_access",
+            "read_only_limited_user"
+          ]
+        },
+        "url": {
+          "icon": "DefaultProperty",
+          "type": "string",
+          "format": "url",
+          "title": "User URL"
+        },
+        "job_title": {
+          "title": "Job Title",
+          "icon": "DefaultProperty",
+          "type": "string"
+        },
+        "contact_methods": {
+          "title": "Contact Methods",
+          "icon": "DefaultProperty",
+          "type": "array"
+        },
+        "description": {
+          "type": "string",
+          "title": "Description"
+        },
+        "teams": {
+          "title": "Teams",
+          "icon": "DefaultProperty",
+          "type": "array"
+        },
+        "time_zone": {
+          "icon": "DefaultProperty",
+          "type": "string",
+          "title": "Time Zone"
+        },
+        "email": {
+          "type": "string",
+          "title": "Email",
+          "format": "user"
+        }
+      },
+      "required": []
+    },
+    "mirrorProperties": {},
+    "calculationProperties": {},
+    "aggregationProperties": {},
+    "relations": {}
+  },
   {
     "identifier": "pagerdutyService",
     "description": "This blueprint represents a PagerDuty service in our software catalog",
@@ -151,6 +216,10 @@
           "title": "Updated At",
           "type": "string",
           "format": "date-time"
+        },
+        "triggered_by": {
+          "type": "string",
+          "title": "Triggered By"
         }
       },
       "required": []

integrations/pagerduty/.port/resources/port-app-config.yaml

@@ -25,6 +25,10 @@ resources:
       apiQueryParams:
         include:
           - assignees
+          - first_trigger_log_entries
+        statuses:
+          - triggered
+          - acknowledged
     port:
       entity:
         mappings:
@@ -41,6 +45,34 @@ resources:
             updated_at: .updated_at
             priority: if .priority != null then .priority.summary else null end
             description: .description
+            triggered_by: .first_trigger_log_entry.agent.summary
+          relations:
+            pagerdutyService: .service.id
+  - kind: incidents
+    selector:
+      query: 'true'
+      apiQueryParams:
+        include:
+          - assignees
+          - first_trigger_log_entries
+        statuses:
+          - resolved
+    port:
+      entity:
+        mappings:
+          identifier: .id | tostring
+          title: .title
+          blueprint: '"pagerdutyIncident"'
+          properties:
+            status: .status
+            url: .html_url
+            urgency: .urgency
+            escalation_policy: .escalation_policy.summary
+            created_at: .created_at
+            updated_at: .updated_at
+            priority: if .priority != null then .priority.summary else null end
+            description: .description
+            triggered_by: .first_trigger_log_entry.agent.summary
           relations:
             pagerdutyService: .service.id
   - kind: schedules
@@ -92,3 +124,21 @@ resources:
             description: .summary
             primaryOncall: .__oncall_users | sort_by(.escalation_level) | .[0].user.email
             escalationRules: .escalation_rules
+  - kind: users
+    selector:
+      query: "true"
+    port:
+      entity:
+        mappings:
+          identifier: .id
+          title: .name
+          blueprint: '"pagerdutyUser"'
+          properties:
+            url: .html_url
+            time_zone: .time_zone
+            email: .email
+            description: .description
+            role: .role
+            job_title: .job_title
+            teams: .teams
+            contact_methods: .contact_methods

integrations/pagerduty/CHANGELOG.md

@@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 <!-- towncrier release notes start -->
 
+## 0.1.121 (2024-11-25)
+
+
+### Improvements
+
+- Improved on the integration default resources for PagerDuty incidents kind
+- Added blueprint and mapping for PagerDuty users
+- Added logs to track the HTTP requests made to PagerDuty API
+- Added a generic resync event handler to allow the integration to bring any valid resource from PagerDuty
+
+
 ## 0.1.120 (2024-11-25)
 
 

integrations/pagerduty/clients/pagerduty.py

@@ -11,6 +11,7 @@
 USER_KEY = "users"
 
 MAX_CONCURRENT_REQUESTS = 10
+PAGE_SIZE = 100
 
 
 class PagerDutyClient:
@@ -77,9 +78,17 @@ async def paginate_request_to_pager_duty(
         has_more_data = True
 
         while has_more_data:
+            logger.debug(
+                f"Fetching data for {resource} with offset: {offset} limit: {PAGE_SIZE} and params: {params}"
+            )
             try:
                 data = await self.send_api_request(
-                    endpoint=resource, query_params={"offset": offset, **(params or {})}
+                    endpoint=resource,
+                    query_params={
+                        "offset": offset,
+                        "limit": PAGE_SIZE,
+                        **(params or {}),
+                    },
                 )
                 yield data[resource]
 
@@ -91,6 +100,11 @@ async def paginate_request_to_pager_duty(
                     f"Got {e.response.status_code} status code while fetching paginated data: {str(e)}"
                 )
                 raise
+            except httpx.HTTPError as e:
+                logger.error(
+                    f"Got an HTTP error while fetching paginated data for {resource}: {str(e)}"
+                )
+                raise
 
     async def get_singular_from_pager_duty(
         self, object_type: str, identifier: str

integrations/pagerduty/integration.py

@@ -24,6 +24,15 @@ class ObjectKind:
     ESCALATION_POLICIES = "escalation_policies"
 
 
+OBJECTS_WITH_SPECIAL_HANDLING = [
+    ObjectKind.SERVICES,
+    ObjectKind.INCIDENTS,
+    ObjectKind.SCHEDULES,
+    ObjectKind.ONCALLS,
+    ObjectKind.ESCALATION_POLICIES,
+]
+
+
 class PagerdutyServiceAPIQueryParams(BaseModel):
     include: (
         list[
@@ -203,9 +212,8 @@ class PagerdutyPortAppConfig(PortAppConfig):
         | PagerdutyScheduleResourceConfig
         | PagerdutyOncallResourceConfig
         | PagerdutyEscalationPolicyResourceConfig
-    ] = Field(
-        default_factory=list
-    )  # type: ignore
+        | ResourceConfig
+    ] = Field(default_factory=list)
 
 
 class PagerdutyIntegration(BaseIntegration):

integrations/pagerduty/main.py

@@ -10,6 +10,7 @@
 from clients.pagerduty import PagerDutyClient
 from integration import (
     ObjectKind,
+    OBJECTS_WITH_SPECIAL_HANDLING,
     PagerdutyEscalationPolicyResourceConfig,
     PagerdutyIncidentResourceConfig,
     PagerdutyOncallResourceConfig,
@@ -175,6 +176,28 @@ async def on_escalation_policies_resync(kind: str) -> ASYNC_GENERATOR_RESYNC_TYP
             yield escalation_policies
 
 
+@ocean.on_resync()
+async def on_global_resync(kind: str) -> ASYNC_GENERATOR_RESYNC_TYPE:
+
+    if kind in OBJECTS_WITH_SPECIAL_HANDLING:
+        logger.info(f"Kind {kind} has a special handling. Skipping...")
+        return
+    else:
+        pager_duty_client = initialize_client()
+
+        try:
+            async for (
+                resource_batch
+            ) in pager_duty_client.paginate_request_to_pager_duty(resource=kind):
+                logger.info(f"Received batch with {len(resource_batch)} {kind}")
+                yield resource_batch
+        except Exception as e:
+            logger.error(
+                f"Failed to fetch {kind} from Pagerduty due to error: {e}. For information on supported resources, please refer to our documentation at https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/incident-management/pagerduty/#supported-resources"
+            )
+            raise e
+
+
 @ocean.router.post("/webhook")
 async def upsert_incident_webhook_handler(data: dict[str, Any]) -> None:
     pager_duty_client = initialize_client()

integrations/pagerduty/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "pagerduty"
-version = "0.1.120"
+version = "0.1.121"
 description = "Pagerduty Integration"
 authors = ["Port Team <[email protected]>"]