Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add oncreated worker #481

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

GwiYeong
Copy link

@GwiYeong GwiYeong commented Dec 5, 2024

I wanted to add a cpu limit for the worker process created when utilizing workerpool in process mode.
However, the current callback doesn't know the pid of the created worker.

This PR adds a callback to get the information of the created worker after the worker is created.
I added a callback called onCreatedWorker, which is passed the currently created WorkerHandler directly.
If it's a process worker, I can get the created pid via worker.worker.pid.
The pid can then be used to limit resources, such as cpu limit.

For cpulimit worker, I add an example.

@josdejong
Copy link
Owner

Thanks @GwiYeong. Thinking aloud here: can we already achieve this already by using events? Would it be possible to let a worker send it's pid to the main process via workerpool.workerEmit?

@GwiYeong
Copy link
Author

@josdejong Thank you for responding.
I just wonder that is there any way to receive messages from a worker that are passed to the workerpool.workerEmit function without calling pool.exec?
In case of cpu limit, it should be called only once per worker, and if the worker is terminated and recreated, it needs to be executed again.
Is there a way to send a message to workerpool.workerEmit as soon as the worker is created, depending on the life cycle of the worker, and have the main process receive and handle it?
If it is possible, can you give me some example code to achieve it?

@josdejong
Copy link
Owner

You can indeed only send/receive workerpool.workerEmit whilst executing a task. But a new worker will only be created when there is a task to execute. You can do do something like this:

// myWorker.js
var workerpool = require('workerpool')

var initialized = false

function wrapperSendPidOnce(fn) {
  return (...args) => {
    if (!initialized) {
      initialized = true

      workerpool.workerEmit({ type: 'created', pid: process.pid })
    }

    return fn(...args)
  }
}

function add(a, b) {
  return a + b
}

function multiply(a, b) {
  return a * b
}

workerpool.worker({
  add: wrapperSendPidOnce(add),
  multiply: wrapperSendPidOnce(multiply)
})

And then use the worker like so:

// main.js
var workerpool = require('workerpool')

var pool = workerpool.pool(__dirname + '/myWorker.js')

function execWithEventListener(method, args) {
  function handleEvent(event) {
    console.log('Event: ', event)
  }

  return pool.exec(method, args, { on: handleEvent })
}

async function run() {
  console.log(await execWithEventListener('add', [2, 3]))
  console.log(await execWithEventListener('add', [7, 2]))
}

run()
  .catch(function (err) {
    console.error(err)
  })
  .finally(function () {
    pool.terminate()
  })

Running this example wil output:

Event:  { type: 'created', pid: 26312 }
5
9

Would that address your case?

@GwiYeong
Copy link
Author

@josdejong yes. I understand what you are saying.
However, It would be easier if we had a lifecycle hook like onCreatedWorker, but it seems a bit complicated to determine whether it's the first time or not.
If a worker's functions should always be executed by calling
How about adding a function like pool.broadcast that triggers all the worker's functions and handles all workers emit?
If so, we could separate functions in the worker that return only the pid and for action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants