Giters

aerokube / moon

Browser automation solution for Kubernetes and Openshift supporting Selenium, Playwright, Puppeteer and Cypress

Home Page:http://aerokube.com/moon/latest

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Not able to enable video recording

buettner123 opened this issue · comments

commented

Hi there,

I'm currently evaluating moon for running our tests in kubernetes and struggling to get the video recording running.

I'm using the moon2 helm chart with version 2.5.4 on an amazon eks cluster.

What I did:

  • setup a bucket
  • create an IAM policy for the bucket access:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:List*",
        "s3:Get*",
        "s3:Put*"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::${TargetBucket}/*",
        "arn:aws:s3:::${TargetBucket}"
      ]
    }
  ]
}
  • allow the service account to assume the IAM policy by a role with the attached policy from above and the following trust relationship (working fine for other helm charts with the same setup):
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::xxxxx:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/xxxxxx"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringLike": {
                    "oidc.eks.eu-central-1.amazonaws.com/id/xxxxxx:sub": [
                        "system:serviceaccount:moon:aerokube-moon",
                        "system:serviceaccount:moon-*:aerokube-moon"
                    ]
                }
            }
        }
    ]
}
  • configure the video recording in the values file for the helm chart:
configs:
  default:
    serviceAccountName: aerokube-moon
    storage:
      bucket: "xxxxxx" # real bucket name masked
      endpoint: "https://s3.eu-central-1.amazonaws.com"
quota:
  moon: null
  test:
    namespace: moon-test
    password: ''
    serviceAccountName: aerokube-moon
  • configure video recording by setting the capabilities:
    desiredCapabilities.setCapability("enableVideo", true)
    desiredCapabilities.setCapability("enableVNC", true)
    desiredCapabilities.setCapability("videoName", GebBasicSpec.caller + ': ' + GebBasicSpec.testName.methodName + ".mp4")
    desiredCapabilities.setCapability("pattern", '$quota/$sessionId')

The test execution works fine and the VNC is enabled successful, but no video is recorded and I can see that for the browserpods no video-recorder pod is created and if I describe the browserpods, I can see that the label "enableVideo" is set to false.

I don't know how to continue here and where to start debugging, as there is no hint how to figure out whats missing in the documentation about the s3 configuration.

Hope you can help. If you need any more details, let me know!

commented

@buettner123 actually these capabilities should go under moon:options key. I.e. you create a capability called moon:options and then put a HashMap<String, Object> inside with all capabilities from your example. https://github.com/aerokube/moon-cloud-java-example/blob/master/src/test/java/com/aerokube/moon/MoonCloudExampleTest.java#L25-L31

commented

Thanks for the hint @vania-pooh. This helped to get the video recording starting.

However I encountered some AccessDenied issues with the saving to the bucket. I think there might be some missing/incomplete documentation for the serviceAccount stuff.

Looking at the values file for the helm chart, I see that it suggest adding:

configs:
  default:
    serviceAccountName: aerokube-moon
    serviceAccountAnnotations: {...}

However, digging into the helm chart I found out that this is actually supposed to be placed on the quota level and the one on config.default.serviceAccount* level doesn't seem to have any effect.

My values.yml now looks like this:

quota:
  moon: null
  test:
    namespace: moon-test
    password: ''
    serviceAccountName: aerokube-moon
    serviceAccountAnnotations: {
      "eks.amazonaws.com/role-arn": "arn:aws:iam::xxxx:role/MoonIAMRole"
    }

With this changes and by providing the role-arn to the service account, it works fine for my eks+iam roles and service account setup.

Maybe it makes sense to adjust the default values.yml file to reflect this or add some more context to the documentation in https://aerokube.com/moon/latest/#faq-kubernetes-service-account

And for the sake of completeness, I adapted the above IAM policy a bit to feature less actions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutObject",
        "s3:ListBucket",
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::${TargetBucket}/*",
        "arn:aws:s3:::${TargetBucket}"
      ]
    }
  ]
}

In addition, TargetBucket is replaced with the bucket name on creation of the policy.

commented

@buettner123 started fixing this and can't find an exact location where we suggest adding the following:

configs:
  default:
    serviceAccountName: aerokube-moon
    serviceAccountAnnotations: {...}

Only found correct example. Could you please provide some reference?

commented

For the serviceAccountName there is https://github.com/aerokube/charts/blob/master/moon2/values.yaml#L350 which seems to have no effect.
The configs.default.serviceAccountAnnotations was an assumption from my side not reflected in the values file (but would make sense in my opinion, if there will be some defaults available)

commented