But also, if I fork the server, where central control can be applied, nobody else will be on my server. The client is only half the problem, and the most insignificant part in my view. I would rather have proprietary client with decentralized infrastructure than the other way round if I could only choose one of the two.

They way signal thinks is, you either are full first party or full ex communicado. You need to set up everything of your own