Find solution for dependency management and release process
weilbith opened this issue · comments
The raiden package is a dependency for the raiden-services. Via this link, the services also access the raiden-contracts which are dependency of raiden itself. Furthermore the services use a big bunch of "utility" features from the raiden package. When a new version of the raiden-contracts gets released, the raiden package gets updated and released too and then the raiden-services can reference that new version to finally get access to the most recent contracts.
Now the raiden package will not become updated anytime soon to make it compatible with changes in the contracts. Only the LightClient implementation will do. But also the raiden-services need to be continued. But due to the raiden dependency, it can't simply update its sub-dependency to the raiden-contracts (would result into a conflict).
The current ugly workaround is to create a git-tag for a commit on the raiden repository that will only update the raiden-contracts version in its requirements.txt file. But these changes will not become released. The raiden-services do then no more reference to a raiden release version as dependency, but a git-tag reference and make a release with that.
This approach isn't really nice. But it is even worse taking into account that this approach needs to be continued in any future as long as the raiden package itself will not see updates to become compatible with the contracts again. So we do releases, but no real releases. Just to satisfy the dependency requirement.
There are at least two approaches to escape solve this problem. The first one is to export the utility functionality of raiden to a new package and have the services reference this and a the contracts separately. Alternatively it must be ensured that the services can specify the contract addresses and the ABI individually, without seeking them indirectly through raiden. This probably requires an extension (making it more generic) of the raiden utility functions like the ContractsManager.
To get an overview of what it would take to remove the raiden dependency from the services, here is the list of the used raiden imports:
$ findimports -n src | grep '^ raiden[.]' | sort -u
raiden.api.v1.encoding.CapabilitiesSchema
raiden.blockchain.filters.decode_event
raiden.constants.DeviceIDs
raiden.constants.EMPTY_SIGNATURE
raiden.constants.Environment
raiden.constants.ETH_GET_LOGS_THRESHOLD_FAST
raiden.constants.ETH_GET_LOGS_THRESHOLD_SLOW
raiden.constants.MatrixMessageType
raiden.constants.Networks
raiden.constants.UINT256_MAX
raiden.exceptions.InvalidSignature
raiden.exceptions.SerializationError
raiden.exceptions.TransportError
raiden.messages.abstract.Message
raiden.messages.abstract.SignedMessage
raiden.messages.monitoring_service.RequestMonitoring
raiden.messages.monitoring_service.SignedBlindedBalanceProof
raiden.messages.path_finding_service.PFSCapacityUpdate
raiden.messages.path_finding_service.PFSFeeUpdate
raiden.network.transport.matrix.client.GMatrixClient
raiden.network.transport.matrix.client.MatrixMessage
raiden.network.transport.matrix.client.node_address_from_userid
raiden.network.transport.matrix.UserPresence
raiden.network.transport.matrix.utils.address_from_userid
raiden.network.transport.matrix.utils.AddressReachability
raiden.network.transport.matrix.utils.DisplayNameCache
raiden.network.transport.matrix.utils.login
raiden.network.transport.matrix.utils.make_client
raiden.network.transport.matrix.utils.ReachabilityState
raiden.network.transport.matrix.utils.UNKNOWN_REACHABILITY_STATE
raiden.network.transport.matrix.utils.UserPresence
raiden.network.transport.matrix.utils.USER_PRESENCE_TO_ADDRESS_REACHABILITY
raiden.network.transport.matrix.utils.validate_user_id_signature
raiden.network.transport.matrix.utils.validate_userid_signature
raiden.network.transport.utils.timeout_exponential_backoff
raiden.settings.DEFAULT_MATRIX_KNOWN_SERVERS
raiden.settings.DEFAULT_NUMBER_OF_BLOCK_CONFIRMATIONS
raiden.settings.DEFAULT_TRANSPORT_MATRIX_RETRY_INTERVAL_INITIAL
raiden.settings.DEFAULT_TRANSPORT_MATRIX_RETRY_INTERVAL_MAX
raiden.settings.DEFAULT_TRANSPORT_MATRIX_SYNC_LATENCY
raiden.settings.DEFAULT_TRANSPORT_MATRIX_SYNC_TIMEOUT
raiden.settings.DEFAULT_TRANSPORT_RETRIES_BEFORE_BACKOFF
raiden.storage.serialization.serializer.JSONSerializer
raiden.storage.serialization.serializer.MessageSerializer
raiden.tests.utils.mediation_fees.get_amount_with_fees
raiden.transfer.mediated_transfer.mediation_fee.FeeScheduleState
raiden.utils.cli.ChainChoiceType
raiden.utils.cli.EnumChoiceType
raiden.utils.cli.GasPriceChoiceType
raiden.utils.cli.get_matrix_servers
raiden.utils.gevent.spawn_named
raiden.utils.keys.privatekey_to_address
raiden.utils.profiling.sampler.FlameGraphCollector
raiden.utils.profiling.sampler.SignalSampler
raiden.utils.signer.LocalSigner
raiden.utils.signer.recover
raiden.utils.typing
raiden.utils.typing.AdditionalHash
raiden.utils.typing.Address
raiden.utils.typing.Balance
raiden.utils.typing.BalanceHash
raiden.utils.typing.BlockNumber
raiden.utils.typing.BlockTimeout
raiden.utils.typing.ChainID
raiden.utils.typing.ChannelID
raiden.utils.typing.FeeAmount
raiden.utils.typing.List
raiden.utils.typing.MonitoringServiceAddress
raiden.utils.typing.MYPY_ANNOTATION
raiden.utils.typing.Nonce
raiden.utils.typing.Optional
raiden.utils.typing.PaymentAmount
raiden.utils.typing.PaymentWithFeeAmount
raiden.utils.typing.PeerCapabilities
raiden.utils.typing.Set
raiden.utils.typing.Signature
raiden.utils.typing.TokenAddress
raiden.utils.typing.TokenAmount
raiden.utils.typing.TokenNetworkAddress
raiden.utils.typing.TransactionHash
Next question would be: what of these are generic enough to be protocol agnostic/contract independent.
Alright, we have decided that for the scope of the Arbitrum project, we have not enough time and money to do it properly. Therefore we will go with the simple tag solution right now. Despite that I will create a new issue for tackling the actual solution, should mean to extract shared code from the raiden repository. This issue will then be scheduled after Arbitrum in the next maintenance phase with its own budget.