Facing communication issue in transmitting data from client to server via SOME/IP between FRANCA Common API C++ Server and Python pysomeip client
Shruthe-96 opened this issue · comments
Hello all,
I am working on establishing inter-process communication between a CommonAPI Some/IP C++ server (Which uses FRANCA IDL and associated deployment files) and a Some/IP python client.
Python Some/IP client
Library used for python Some/IP client:
someip 0.1.0, Pysomeip (https://pypi.org/project/someip/ )
import someip.header
from someip.sd import SOMEIPDatagramProtocol
Franca C++ Server
The C++ server that is used was setup as per the following link:
https://github.com/GENIVI/capicxx-core-tools/wiki
Work done till now:
(C++ vsomeip server) <-------------------------- > (python client) Result: RPC Worked here.
Tried Some/IP IPC using a basic C++ vsomeip server (Which does not include CommonAPI and Franca) and the python client.
Here, both client and server are able to read each other’s message.
Server C++
Used the “Request/Response” example from the following link as the C++ server:
https://github.com/GENIVI/vsomeip/wiki/vsomeip-in-10-minutes
Snapshot 1 having log message on the Server Terminal Side.
Client Python
And the following example as the python client:
https://github.com/afflux/pysomeip/blob/master/tools/get.py
Snapshot 2 have the snapshot at the python side terminal log
I am attaching the snapshot of the IPC between the server and client:
Server:
Fig: Snapshot 1
Client:
Fig: Snapshot 2
Issue Details:
On using the same client code to establish communication with the C++server code (Containing franca and commonAPI as mentioned in https://github.com/GENIVI/capicxx-dbus-tools/wiki/CommonAPI-C---D-Bus-in-10-minutes#step5)
RPC between the client and server works only when we send ‘\x00’ as payload from the client to the server.
The output at the server and client are as shown in snapshot 3:
Server:
Client:
Fig: Snapshot 3
The server here uses a .fidl file wherein the data type of the ‘in’ parameter as ByteBuffer (whose C++ equivalent is std::vector<uint8_t>).
Franca IDL supports the following data types : int, Bool, float, Double, string and std::vector<uint8_t>
-
Is it possible to send data of different data types other than bytes. As all data that I tried sending – String, int etc. always get converted into bytes in hdr.build(). Is difference in the data type a reason the server is unable to read the client's message?
-
Also, is it possible to create a SOME/IP python RPC/Broadcast server using pysomeip?
-
https://github.com/afflux/pysomeip/blob/32b5b20bfc923ef7f1aad51732da058f53ac8f66/tools/find-subscribe.py - Here, how is the subscription request sent to the server as there is not ‘send’ method here. When I tried to execute this code it resulted in the following error as shown in Snapshot 4.
Fig: Snapshot 4
Kindly help me in solving this isssue. Thanks in advance!
Re 1.: not sure if I understand your question. Of course the data is transmitted as bytes. If your application uses CommonAPI for serialization, you'll need to deal with that. Pass the serialized data as SOMEIPHeader(payload=...). Same goes the other way round: when you receive a message, you can run your CommonAPI deserialization on msg.payload.
Re 2.: of course. Right now you'll have to do a lot of leg-work yourself, however. You'd have to create a local endpoint for your service (subclass SOMEIPDatagramProtocol, call create_unicast_endpoint()). Every incoming message to your service (method calls, field gets, field sets) will call message_received on the protocol instance. You'd need to check the messages' fields and react accordingly. You'd also have to set up a ServiceDiscoveryProtocol, call announcer.announce_service() with your someip.config.Service and a ServerServiceListener. The ServerServiceListener will get callbacks whenever a client subscribes or unsubscribes to eventgroups. I'm in the process of implementing a few standard interfaces for this, but I cannot give you an ETA.
Re 3.: I'll have to fix this. I think I changed the API didn't adapt the sample code. I'll keep the issue open for now.
Thank you very much for taking time to help me out sir.
Sorry for being unclear in Question 1 - I meant to ask if it is possible to send different data types such as Integer, String etc. in (payload=...), For example: payload = 'Bob'. As in my case the server is expecting to receive a 'String' from the client.
Let me re-iterate: this library currently does not care how you structure / encode your payload data. That's a different abstraction layer.
There are many Python libraries that deal with serialization, some more basic and flexible (struct, construct) - allowing you to implement any custom serialization format, some more specific to certain use cases (protobuf)
If you feel there is a common serialization format that's specific to SOME/IP that you need, feel free to implement it in a separate module and open a pull request.
Thank you so much for your advice sir.
Hello,
I was able to read the value sent from the python client at the c++ server's side and vice-versa. Thank you very much for your advice sir. I had to change the data type at the server side to 'Int64_t' and it worked as expected. I initialized the payload as shown below:
- Tried transmitting an integer value as payload=b"x\01" which the server was able to receive
- Tried transmitting a string as payload=b"hey", in this case I only had to typecast the integer value to char at the server side and it worked as expected
Now, I have begun working on verifying broadcast communication between the C++ server and python client. The server sends out UDP broadcast messages. I have used "find-subscribe.py" from the tools folder as a reference. However, the python client is unable to receive any of the broadcast sent from the server as the client is not subscribing to the event-group. As mentioned in my previous question the following error still persists:
I commented out the lines protocol.start() and protocol.stop() just to check what happens. The code continues to remain in the while loop (line number 73 in https://github.com/afflux/pysomeip/blob/master/tools/find-subscribe.py) and doesn't seem to receive any of the broadcast.
To check if the server is actually sending out the broadcasts and also to check if i was using the right broadcast group address and port i used tcpdump to sniff and found that the broadcasts are being sent out by the server. I also used Wireshark to double-check. The snapshots are as below:
Is it possible to receive broadcast messages by subscribing to the event group using pysomeip library or is it yet to be developed?
Also, in this particular reply (#2 (comment)) you had mentioned that you have to adapt the changes in the sample code, is it done yet sir? As it would be very useful for me to verify broadcast communication between python and C++.
Kindly help me on how to go about this.
Had a quick look at the AttributeError that you're seeing. I don't see how this can happen, as ServiceDiscoveryProtocol.start() and ServiceDiscoveryProtocol.stop() are both implemented.
(Obviously, if .start() isn't working, and you're commenting it out, the whole FindService / SubscribeEventGroup sequence will not start, so it's correct that you're not receiving notifications.)
As I am unable to reproduce, can you please share a minimum example that exhibits the AttributeError, so I can verify and debug this?
Just simply executing the code "find-subscribe.py" with all the required parameters - local address, multicast address, local port number and event Id resulted in the attribute error.
I executed it using :
python3.8 find-subscribe.py 0.0.0.0 224.244.224.245 31000 4660 33010
Also, in the main() function I set the instance = 22136 and major_version=0x01.
Thanks. Turns out, the release on PyPI somehow was not built with the 0.1.0 tag, therefore it did not include eacff98. I've just published 0.1.1 on PyPI, this should fix your issue.
Hi sir, The code is working now. I would like to brief on what I'm doing now:
Currently I'm broadcasting an integer value from the C++ server and trying to capture it via the client. A snapshot of the server is as below:
The json file at the server side is configured as below:
And the client is similar to - https://github.com/afflux/pysomeip/blob/master/tools/find-subscribe.py
I'm getting the following payload at the client side:
But it doesn't seem to contain the exact value (in my case 26) at the payload.
I have tried transmitting value of different datatypes via broadcast (string, hex values, byetbuffer etc.) from the server but the payload received at the client side is consistently this same value.
I have been trying different approaches and figure out why this is happening. A few observations:
- The eventgroup ID, instance ID and service ID i've been using at the client is same as that at the server.
- Since the client is receiving some kind of a payload it must be subscribing to the correct service.
- I've also verified all the udp ports using 'ss -lntu' and captured the packets in these ports using tcpdump. Only the port that I've configured at the server side for service discovery seems to transmit multicast message. So, I confirmed that i've provided the right port and address at the client side. A snapshot of the tcpdump output is as below:
I will continue debugging the code and keep you updated when it works.
Any suggestions from your side as to why the payload doesn't seem to contain the exact value being transmitted and remains the same value no matter what data is being broadcasted from the server side would be very much appreciated.
Thanks a lot for your prompt replies.
The messages that you are printing on your client side are SD messages, but you're treating these as if they were your event group notifications. My first guess would be wrong ports on your client (30490 is SD, nothing else), but it's impossible to say without seeing your client code.
Sure sir!
The following is the code snippet where i've initialized details related to subscription of eventgroup:
Here, since i wanted to combine bith RPC and broadcast, I've included prot.get() in the try block. Also, I want to continuously check and receive broadcasted values i've added it inside the infinite loop. Here, at the event group receiver I have initialised the local port as 30490 as below:
Also, when changed the local port to 31000 as below:
The following is the output:
I receive a negative subscription acknowledgment.
I think the subscribe NACK is coming from your own SD service.
I've pushed 6c3997d to the no-loop branch to test without loopback. Please test by installing with pip install git+https://github.com/afflux/pysomeip.git@no-loop
I tried with the version you mentioned above sir. Still, the NACK exists and I'm not receiving the eventgroup messages. The log is as below:
For the moment I have removed the RPC portion of te code to make the log look less clumsy. Client:
Thanks for testing the branch. Can you please try to run each side on a different host? (Or create a pair of veth devices in a private IP range, move one of them to a different net namespace and run one side of your experiment in that namespace.) Both SD instances are bound to the same local port with SO_REUSEPORT, which leads to the kernel distributing the incoming packets evenly between both - this will definitely cause problems.
No worries sir! I will try that and get back to you. Thank you very much.
Hi sir, Sorry for the late update. I ran the server and client in two different network namespaces and both RPC as well as broadcast seemed to work as expected. Here is a screenshot:
Thank you for the suggestions!