feat: add static init for instances of Partition and Authentication

.docs/content/1.concepts/3.authentication.md

@@ -35,7 +35,7 @@ When an user sends a request to an endpoint, they need to be authenticated, and
 
 ## Request authorization flowchart
 
-<Mermaid>
+```mermaid
 flowchart TB
     User([User Certificate])
     OK(OK)
@@ -56,85 +56,93 @@ flowchart TB
     CheckPerm --> |No|DENIED
     CheckPerm --> |Yes|OK
     Cache --> |Yes|CheckPerm
-</Mermaid>
+```
 
 ## User administration
 
-Users, roles, permissions and certificates are stored and managed in a database. Administrators in charge of handling user permissions can refer to this section to manage user permissions.
-
-### Authentication database deployments
-
-The following database deployments can be used to handle authentication data :
-
-- Internal MongoDB
-- External MongoDB (Currently not supported)
-- External SQL (Currently not supported)
+Users, roles, permissions and certificates are stored and managed by ArmoniK via environment variables provided to the control plane and compute plane. Administrators in charge of handling user permissions can refer to this section to manage user permissions.
 
 ### Populating the internal MongoDB when deploying ArmoniK
 
 > **NOTE :** This is the current recommended method
 
 You can define the users' roles and certificates using a json configuration during deployment. Check the [ArmoniK Authentication Configuration Guide](https://github.com/aneoconsulting/ArmoniK/blob/main/.docs/content/2.guide/1.how-to/how-to-configure-authentication.md) for more details.
 
-### Using MongoDB directly
-
-<!-- TODO: could be converted to ::alert -->
-> **NOTE :** Using a MongoDB instance different from the one used in the rest of ArmoniK is currently not supported
-> **NOTE :** If the deployment used is the internal MongoDB, to send commands to the database you need to use a script like the one available [here](https://github.com/aneoconsulting/ArmoniK/blob/main/tools/access-mongo-as-user.sh) to connect to the database as a user. Another script to connect to the database as admin is available [here](https://github.com/aneoconsulting/ArmoniK/blob/main/tools/access-mongo-as-admin.sh).
-> **NOTE :** This method is available to anyone having access to the deployed cluster's secrets and an access to the MongoDB host and port. **Use it at your own risk**.
+### Using ArmoniK environment variables
 
-In order to function properly, the MongoDB database needs to have the following collections:
+In order to function properly, the authentication needs to have the following collections:
 
-- AuthData
+- List of [Certificate](../../../Common/src/Injection/Options/Database/Certificate.cs)
   - Handles the association between certificates and users
   - Requires the following fields:
-    - UserId : _id field of the UserData object associated with this certificate
+    - User : Name field of the User object associated with this certificate
     - CN : Certificate's Common name
     - Fingerprint : null if this entry should match all certificates with the given CN, otherwise, the certificate's fingerprint
   - The CN and Fingerprint fields form a unique compound index.
-- UserData
+- List of [User](../../../Common/src/Injection/Options/Database/User.cs)
   - Handles the association between a user and its roles
   - Requires the following fields
-    - Username : Unique user name
+    - Name : Unique user name
     - Roles : list of objectIds, each matching the _id field in RoleData of the roles given to the user
-- RoleData
+- List of [Role](../../../Common/src/Injection/Options/Database/User.cs)
   - Handles the association between a role and its permissions
   - Requires the following fields
-    - RoleName : Unique role name
+    - Name : Unique role name
     - Permissions : list of strings corresponding to the permissions of the role
 
-#### Insert a role
+These collections of object needs to be provided as JSON objects as detailled in the following sections.
 
-To insert a role with the name "Role1" granting the permissions "Submitter:ListTasks" and "General:Impersonate:Role2", use the following command :
+#### Environment variables base
 
-```javascript
-db.RoleData.insertOne({RoleName:"Role1", Permissions:["Submitter:ListTasks", "General:Impersonate:Role2"]})
+An InitServices options class was introduced to initialize services.
+It contains two classes : Authentication and Partitionning to configure authentications and Partitions respectively.
+Authentication has several list of strings as fields: UserCertificates, Roles and Users.
+Those fields are JSON strings that are deserialized into corresponding objects that will be inserted into the database.
+
+These options can be configured as environment variables.
+Lists should be converted into an environment variable per element with its index as shown below.
+
+For example, the environment variables for the first and second roles are:
+
+```bash
+InitServices__Authentication__Roles__0=<json string containing a [Role]>
+InitServices__Authentication__Roles__1=<json string containing a [Role]>
 ```
 
-See the [MongoDB documentation](https://www.mongodb.com/docs/manual/reference/method/db.collection.insertOne/) to get the RoleId (_id field) of the inserted role from the command's response.
+#### Specify roles
 
-#### Insert a user
+To specify a role with the name "Role1" granting the permissions "Submitter:ListTasks" and "General:Impersonate:Role2", use the following environment variable:
 
-To insert a user with the name "User1" with the role "Role1", use the following command :
+```bash
+InitServices__Authentication__Roles__0='{"Name": "Role1", "Permissions": ["Submitter:ListTasks", "General:Impersonate:Role2"]}'
+```
 
-```javascript
-db.UserData.insertOne({User:"User1", Roles:["Role1"]})
+If you need a second role, you can add the following environment variable too:
+
+```bash
+InitServices__Authentication__Roles__1='{"Name": "Role2", "Permissions": ["Submitter:ListTasks", "General:Impersonate:Role3"]}'
 ```
 
-See the [MongoDB documentation](https://www.mongodb.com/docs/manual/reference/method/db.collection.insertOne/) to get the UserId (_id field) of the inserted user from the command's response.
+#### Specify users
 
-#### Insert a certificate
+To specify a user with the name "User1" with the role "Role1", use the following command :
+
+```bash
+InitServices__Authentication__Users__0='{"Name": "User1", "Roles": ["Role1"]})'
+```
 
-To insert a certificate with Common Name "CN1" and Fingerprint "FP1" associated with the user with UserId "62f4efe6d82645e26e09584f", use the following command :
+#### Specify certificates
+
+To insert a certificate with Common Name "CN1" and Fingerprint "FP1" associated with the user with the User called "User1", use the following command :
 
 ```javascript
-db.AuthData.insertOne({UserId:"62f4efe6d82645e26e09584f", CN:"CN1", Fingerprint:"FP1"})
+InitServices__Authentication__UserCertificates__0='{"User": "User1", "CN": "CN1", "Fingerprint": "FP1"}'
 ```
 
 To insert an entry matching all certificates with Common Name "CN1" associated with user with UserId "62f4efe6d82645e26e09584f", use the following command :
 
 ```javascript
-db.AuthData.insertOne({UserId:"62f4efe6d82645e26e09584f", CN:"CN1"})
+InitServices__Authentication__UserCertificates__0='{"User": "User1", "CN": "CN1"}'
 ```
 
 #### Edit a user/role/certificate
@@ -145,10 +153,6 @@ Refer to [this link](https://www.mongodb.com/docs/manual/reference/method/db.col
 
 Refer to [this link](https://www.mongodb.com/docs/manual/reference/method/db.collection.findOneAndDelete/) for the procedure to delete MongoDB entries.
 
-### Using SQL directly
-
-Currently not implemented
-
 ### Using the User Administration gRPC service
 
 Currently not implemented

.github/workflows/build.yml

@@ -421,19 +421,28 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        queue:
-          - activemq
-          - rabbitmq
-          - rabbitmq091
-          - pubsub
-          - sqs
-        object:
-          - redis
-          - minio
-        log-level:
-          - Information
-          - Verbose
-    name: HtcMock ${{ matrix.queue }} ${{ matrix.object }} ${{ matrix.log-level }}
+        target:
+          - { queue: activemq, object: redis, log-level: Information, cinit: true }
+          - { queue: rabbitmq, object: redis, log-level: Information, cinit: true }
+          - { queue: rabbitmq091, object: redis, log-level: Information, cinit: true }
+          - { queue: pubsub, object: redis, log-level: Information, cinit: true }
+          - { queue: sqs, object: redis, log-level: Information, cinit: true }
+
+          - { queue: activemq, object: redis, log-level: Verbose, cinit: true }
+          - { queue: rabbitmq, object: redis, log-level: Verbose, cinit: true }
+          - { queue: rabbitmq091, object: redis, log-level: Verbose, cinit: true }
+          - { queue: pubsub, object: redis, log-level: Verbose, cinit: true }
+          - { queue: sqs, object: redis, log-level: Verbose, cinit: true }
+
+          - { queue: activemq, object: minio, log-level: Information, cinit: true }
+          - { queue: rabbitmq, object: minio, log-level: Information, cinit: true }
+          - { queue: rabbitmq091, object: minio, log-level: Information, cinit: true }
+          - { queue: pubsub, object: minio, log-level: Information, cinit: true }
+          - { queue: sqs, object: minio, log-level: Information, cinit: true }
+
+          - { queue: activemq, object: redis, log-level: Information, cinit: false }
+
+    name: HtcMock ${{ matrix.target.queue }} ${{ matrix.target.object }} ${{ matrix.target.log-level }} ${{ matrix.target.cinit }}
     steps:
       - name: Checkout
         uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
@@ -455,7 +464,7 @@ jobs:
       - name: Deploy Core
         run: |
           MONITOR_PREFIX="monitor/deploy/" tools/retry.sh -w 30 -- tools/monitor.sh \
-            just log_level=${{ matrix.log-level }} tag=${VERSION} queue=${{ matrix.queue }} object=${{ matrix.object }} worker=htcmock deploy
+            just log_level=${{ matrix.target.log-level }} tag=${VERSION} queue=${{ matrix.target.queue }} object=${{ matrix.target.object }} cinit=${{ matrix.target.cinit }} worker=htcmock deploy
           sleep 10
 
       - name: Print And Time Metrics
@@ -524,7 +533,7 @@ jobs:
 
       - name: Run HtcMock test 1000 tasks 1 level
         timeout-minutes: 3
-        if: ${{ matrix.log-level != 'Verbose' }}
+        if: ${{ matrix.target.log-level != 'Verbose' }}
         run: |
           MONITOR_PREFIX="monitor/htcmock-1000-1/" tools/monitor.sh \
             docker run --net armonik_network --rm \
@@ -539,7 +548,7 @@ jobs:
 
       - name: Run HtcMock test 1000 tasks 4 levels
         timeout-minutes: 3
-        if: ${{ matrix.log-level != 'Verbose' }}
+        if: ${{ matrix.target.log-level != 'Verbose' }}
         run: |
           MONITOR_PREFIX="monitor/htcmock-1000-4/" tools/monitor.sh \
             docker run --net armonik_network --rm \
@@ -563,8 +572,8 @@ jobs:
         run: |
           export AWS_ACCESS_KEY_ID=${{ secrets.AWS_ACCESS_KEY_ID }}
           export AWS_SECRET_ACCESS_KEY=${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          docker cp fluentd:/armonik-logs.json - | gzip -c | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.queue }}-${{ matrix.object }}-${{ matrix.log-level }}.json.gz
-          tar -czf - monitor/ | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.queue }}-${{ matrix.object }}-${{ matrix.log-level }}-monitor.tar.gz
+          docker cp fluentd:/armonik-logs.json - | gzip -c | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.target.queue }}-${{ matrix.target.object }}-${{ matrix.target.log-level }}-${{ matrix.target.cinit }}.json.gz
+          tar -czf - monitor/ | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.target.queue }}-${{ matrix.target.object }}-${{ matrix.target.log-level }}-${{ matrix.target.cinit }}-monitor.tar.gz
 
       - name: Collect docker container logs
         uses: jwalton/gh-docker-logs@2741064ab9d7af54b0b1ffb6076cf64c16f0220e # v2
@@ -576,14 +585,14 @@ jobs:
         run: |
           export AWS_ACCESS_KEY_ID=${{ secrets.AWS_ACCESS_KEY_ID }}
           export AWS_SECRET_ACCESS_KEY=${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          tar -cvf - ./container-logs | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.queue }}-${{ matrix.object }}-${{ matrix.log-level }}-container-logs.tar.gz
+          tar -cvf - ./container-logs | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.target.queue }}-${{ matrix.target.object }}-${{ matrix.target.log-level }}-${{ matrix.target.cinit }}-container-logs.tar.gz
       - name: Export and upload database
         if: always()
         run: |
           export AWS_ACCESS_KEY_ID=${{ secrets.AWS_ACCESS_KEY_ID }}
           export AWS_SECRET_ACCESS_KEY=${{ secrets.AWS_SECRET_ACCESS_KEY }}
           bash tools/export_mongodb.sh
-          tar -cvf - *.json | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.queue }}-${{ matrix.object }}-${{ matrix.log-level }}-database.tar.gz
+          tar -cvf - *.json | aws s3 cp - s3://${{ secrets.AWS_LOG_BUCKET_NAME }}/core-pipeline/${{ github.run_number }}/${{ github.run_attempt }}/htcmock-${{ matrix.target.queue }}-${{ matrix.target.object }}-${{ matrix.target.log-level }}-${{ matrix.target.cinit }}-database.tar.gz
 
   testWindowsDocker:
     needs:

Adaptors/MongoDB/src/Common/IMongoDataModelMapping.cs

@@ -17,6 +17,8 @@
 
 using System.Threading.Tasks;
 
+using ArmoniK.Core.Common.Injection.Options.Database;
+
 using MongoDB.Driver;
 
 namespace ArmoniK.Core.Adapters.MongoDB.Common;
@@ -25,10 +27,44 @@ public interface IMongoDataModelMapping<T>
 {
   string CollectionName { get; }
 
+  /// <summary>
+  ///   Setup indexes for the collection
+  ///   Can be called multiple times
+  /// </summary>
+  /// <param name="sessionHandle">MongoDB Client session</param>
+  /// <param name="collection">MongoDDB Collection in which to insert data</param>
+  /// <param name="options">Options for MongoDB</param>
+  /// <returns>
+  ///   Task representing the asynchronous execution of the method
+  /// </returns>
   Task InitializeIndexesAsync(IClientSessionHandle sessionHandle,
                               IMongoCollection<T>  collection,
                               Options.MongoDB      options);
 
+  /// <summary>
+  ///   Setup sharding for the collection
+  ///   Can be called multiple times
+  /// </summary>
+  /// <param name="sessionHandle">MongoDB Client session</param>
+  /// <param name="options">Options for MongoDB</param>
+  /// <returns>
+  ///   Task representing the asynchronous execution of the method
+  /// </returns>
   Task ShardCollectionAsync(IClientSessionHandle sessionHandle,
                             Options.MongoDB      options);
+
+  /// <summary>
+  ///   Insert data into the collection after its creation.
+  ///   Can be called multiple times
+  /// </summary>
+  /// <param name="sessionHandle">MongoDB Client session</param>
+  /// <param name="collection">MongoDDB Collection in which to insert data</param>
+  /// <param name="initDatabase">Data to insert</param>
+  /// <returns>
+  ///   Task representing the asynchronous execution of the method
+  /// </returns>
+  Task InitializeCollectionAsync(IClientSessionHandle sessionHandle,
+                                 IMongoCollection<T>  collection,
+                                 InitDatabase         initDatabase)
+    => Task.CompletedTask;
 }