Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: Integrate reconnect tasks into ChiaServer #14071

Closed

Conversation

xdustinface
Copy link
Contributor

@xdustinface xdustinface commented Dec 7, 2022

Just feels like it belongs in there rather than in its own file because we pass in an ChiaServer anyway and access its connections and starting the client.

Based on:

@xdustinface xdustinface added Changed Required label for PR that categorizes merge commit message as "Changed" for changelog Cleanup Code cleanup labels Dec 7, 2022
@xdustinface xdustinface force-pushed the pr-server-reconnect-tasks branch 2 times, most recently from d3ae22e to 92ed6c7 Compare December 7, 2022 14:58
@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Dec 8, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2022

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@xdustinface xdustinface force-pushed the pr-server-reconnect-tasks branch 3 times, most recently from 07700ea to fcbf301 Compare December 15, 2022 04:54
@xdustinface xdustinface removed the merge_conflict Branch has conflicts that prevent merge to main label Dec 15, 2022
@xdustinface xdustinface force-pushed the pr-server-reconnect-tasks branch from fcbf301 to b9843af Compare December 16, 2022 03:54
@xdustinface xdustinface reopened this Dec 17, 2022
@xdustinface xdustinface force-pushed the pr-server-reconnect-tasks branch 3 times, most recently from f38a55c to 7ce6243 Compare December 17, 2022 16:48
@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Jan 6, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jan 6, 2023

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@xdustinface xdustinface force-pushed the pr-server-reconnect-tasks branch from 7ce6243 to 0d45aa2 Compare February 6, 2023 04:45
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label Feb 6, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2023

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Feb 8, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Feb 8, 2023

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@xdustinface xdustinface force-pushed the pr-server-reconnect-tasks branch from 0d45aa2 to 11f6f80 Compare February 13, 2023 04:41
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label Feb 13, 2023
@github-actions
Copy link
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@xdustinface xdustinface marked this pull request as ready for review February 13, 2023 06:56
@xdustinface xdustinface requested a review from a team as a code owner February 13, 2023 06:56
@arvidn
Copy link
Contributor

arvidn commented Feb 14, 2023

this is not an obvious improvement to me. for example, if we ever were to write unit tests for this function, having it be a free function would make that a lot simpler (we would just need to come up with a protocol to mock the ChiaServer we pass in, or pass in the two members we access separately).

There are a few other changes in here that seems like obvious improvements though (and also somewhat unrelated to moving the free function into the class). for example, moving the exception type into the error file

@xdustinface
Copy link
Contributor Author

this is not an obvious improvement to me. for example, if we ever were to write unit tests for this function, having it be a free function would make that a lot simpler (we would just need to come up with a protocol to mock the ChiaServer we pass in, or pass in the two members we access separately).

Actually it seemed clear to me that it's part of ChiaServer. Since it's not obvious to you, can you explain where you draw the line of making a function accessing internals of a class a method vs. free function?

Not sure why a maybe coming unit test would justify leaving this method in its own file with a public interface vs integrating it as private method into the related class? Also i don't really get how the "pass in the two members separately" would look like with this being a task which access them at runtime (not only on task creation) other than holding a reference to the passed in members in the task which seems risky/unintuitive?

There are a few other changes in here that seems like obvious improvements though (and also somewhat unrelated to moving the free function into the class). for example, moving the exception type into the error file

There are few somewhat unrelated simplifications, right, like getting rid of the inner wrapping method and instead create the task in place or simplify the "are we still connected check". But moving the exception isn't unrelated imo. It's just, there is a ServiceException before and now there is a ServerException because the PR integrated the reconnecting from the service into the server moving the exception is just to not have an exception in the wrong place.

Copy link
Contributor

@arvidn arvidn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is start_service.py the only user of ChiaServer?
By moving this logic from start_service into ChiaServer you seem to suggest that is the case. If not, this is a behavior change

@@ -60,7 +60,7 @@ async def handshake_done() -> bool:
# Start both services and wait a bit
await farmer_service.start()
await harvester_service.start()
harvester_service.add_peer(PeerInfo(str(farmer_service.self_hostname), farmer_service._server.get_port()))
harvester_service._server.add_peer(PeerInfo(str(farmer_service.self_hostname), farmer_service._server.get_port()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accessing the internal member her will make it harder to change implementation details of harvester_service, this increases technical debt. I would prefer tests use the public interface.

def add_peer(self, peer: PeerInfo) -> None:
if self._reconnect_tasks.get(peer) is not None:
raise ServerError(f"Peer {peer} already added")
self._reconnect_tasks[peer] = asyncio.create_task(self._reconnect_task_handler(peer))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you know why we have a reconnect task for each peer? It seems really wasteful for every peer to iterate over all peers just to find itself. It would seem simpler an more efficient to just have one task that iterates over all peers and retries the ones that should be retried

from chia.types.peer_info import PeerInfo


def start_reconnect_task(server: ChiaServer, peer_info: PeerInfo, log: Logger) -> asyncio.Task[None]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nothing this function accesses from server starts with an underscore. I would think all_connections and start_client() are meant to be public

@arvidn
Copy link
Contributor

arvidn commented Feb 20, 2023

Actually it seemed clear to me that it's part of ChiaServer. Since it's not obvious to you, can you explain where you draw the line of making a function accessing internals of a class a method vs. free function?

I think the short answer is: where it makes sense to chop up logic to be unit tested.

Not sure why a maybe coming unit test would justify leaving this method in its own file with a public interface vs integrating it as private method into the related class?

I would say it's increasing the coupling between those two parts. Generally I think we should be moving in the direction of splitting classes into smaller pieces. That makes them much easier to unit test.

Also i don't really get how the "pass in the two members separately" would look like with this being a task which access them at runtime (not only on task creation) other than holding a reference to the passed in members in the task which seems risky/unintuitive?

For example, the peer list could be passed in separately from the function to reconnect a peer. This way it would make a test much simpler because you wouldn't have to create something that looks like a ChiaServer sharing references to mutable data isn't very nice, but it's how it works now. But perhaps more importantly, it doesn't look like a very good idea to have a task per peer for this to begin with.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2023

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Mar 6, 2023
@xdustinface
Copy link
Contributor Author

Closing in favour of #14997.

@xdustinface xdustinface closed this Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changed Required label for PR that categorizes merge commit message as "Changed" for changelog Cleanup Code cleanup merge_conflict Branch has conflicts that prevent merge to main
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants