prysmaticlabs / prysm

Describe the bug

Was looking into ChainSafe/lodestar#6633 compatibility issue of Lodestar BN <> Prsym VC.

The error on the Prysm side

time="2024-06-08 20:13:34" level=info msg="Syncing with beacon node to align on chain genesis info" prefix=client
time="2024-06-08 20:13:34" level=warning msg="Could not determine if beacon chain started" error="could not receive ChainStart from stream: failed to get genesis data: genesis data is nil: could not connect" prefix=client

But the response from Lodestar looks correct

time="2024-06-08T20:13:44Z" level=info msg="REQUEST #2: GET /eth/v1/beacon/genesis" length=0 type=json
time="2024-06-08T20:13:44Z" level=info msg="RESPONSE #2: GET /eth/v1/beacon/genesis" length=169 status=200 type=json
{
  "data": {
    "genesis_fork_version": "0x10000038",
    "genesis_time": "1717877689",
    "genesis_validators_root": "0x70498f18a6f07107a2a05d0d36c7cbd93b3e439023cbef21a6681eebee194526"
  }
}

I believe the reason why the data is nil is due to the way the content-type header is checked

prysm/validator/client/beacon-api/json_rest_handler.go

Line 112 in 8413660

if httpResp.Header.Get("Content-Type") != api.JsonMediaType {

it compares the whole header against the media type but in reality, the conten type header can also include the charset and boundry (see Content-Type#directives).

Lodestar returns the following header

content-type: application/json; charset=utf-8

which is a valid content-type header format, on Prysm side you probably wanna apply something like

strings.SplitN(contentType, ";", 2)[0]

Has this worked before in a previous version?

No response

🔬 Minimal Reproduction

Kurtosis config

participants:
  - el_type: geth
    el_image: ethereum/client-go:stable
    cl_type: lodestar
    cl_image: chainsafe/lodestar:latest
    vc_type: prysm
    vc_image: gcr.io/prysmaticlabs/prysm/validator:latest
    count: 2
  - el_type: geth
    el_image: ethereum/client-go:stable
    cl_type: prysm
    cl_image: gcr.io/prysmaticlabs/prysm/beacon-chain:latest
    vc_type: lodestar
    vc_image: chainsafe/lodestar:latest

Error

No response

Platform(s)

Linux (x86)

What version of Prysm are you running? (Which release)

v5.0.3

Anything else relevant (validator index / public key)?

Related issue ChainSafe/lodestar#6633

Thanks for the report will fix!

All Lodestar <> Prysm issues seem to be resolved 🎉

Running the following config

participants:
  - el_type: geth
    el_image: ethereum/client-go:stable
    cl_type: lodestar
    cl_image: nflaig/lodestar:ignore-empty-statuses
    vc_type: prysm
    vc_image: ethpandaops/prysm-validator:develop-dfe31c9
    count: 2
  - el_type: geth
    el_image: ethereum/client-go:stable
    cl_type: prysm
    cl_image: ethpandaops/prysm-beacon-chain:develop-dfe31c9
    vc_type: lodestar
    vc_image: nflaig/lodestar:ignore-empty-statuses
    vc_extra_params:
      - --http.requestWireFormat=ssz
    count: 2

I can also confirm that #14075 works as expected, we can enable ssz-only mode on Lodestar with Prysm without any errors (as it falls back to JSON as expected now).

Just a quick side note, I noticed another issue that Prysm VC sends an empty statuses array when calling POST /eth/v1/beacon/states/head/validators. I would argue this is inline with the spec and I improved the handling on Lodestar side (ChainSafe/lodestar#6876) but previously this would just result in returning no validators (irrespective of ids passed).

Prysm VC more robust content-type header handling