datamechanics / delight

A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.

Home Page:https://www.datamechanics.co/delight

Repository from Github https://github.comdatamechanics/delightRepository from Github https://github.comdatamechanics/delight

How to decode compressed data

georgehu0815 opened this issue · comments

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Need a solution to decompress the data object on payload.
JField(
// "data",
// JString(Base64.getEncoder.encodeToString(compressString(data)))
// )

we need a scala function to decode the zip compressed Data part. Can you have a util class/method to do that?

{"counters":{"messageCounter":31,"payloadCounter":2},"data":"H4sIAAAAAAAA=="}

class java.lang.String : H4sIAAAAAAAAAO1dW2/iyBJ+31+BLB5mpYjjOzZvGZKZzZwwiSAz+3CyGnVwk3hibMs20WYj/vup7vYNsAmQAaxsvSTuS1X1pbq+j2oSv0jnT9RPpJ40Ckn0eOnGCfVp9CW4GyUkSqQTCR5bF2dSTz6
RRrO7qRvHbuC3btwplXqK2dVtxbDkrmaa0J6Qe9q68CdBLPX+95KVU2FeOE0SOg2TxcqvhCmTIurMxrRFkhYfy7XbicfEIz3NgmF8nU3vaNQKJq0bEj+CfgXEh2dn3By3xgugVoHOQuGAhNcwCTeBEcfQDGpG4yBkTS+3kuvcSr1bSbmVTm4lHyR4cUrCW2kOPfvE82I3YZ2hr
mJUOnQC9bB6YJZNWP6LTSiI2JQu6RP1pN6L9C2mrTM3fpR6E+LF9IRXDOg0iJ7zqjMa08glnvsPdfLKIQ09d0zY2GFO8xPpI4kil0YlqYRGU9eHLXPHqT3p7PzmfDi4+Hp6c76waMU6iJUrWvpk/ECdhQ7QLkbYGsGQeJlNISvNT4rFlvPFBgWwYtTrB/BzzDTVrbi8uOJhKgj
KV1a+1LbJDvxbNuCv1XnDYIjrQUEKovsOCZnSTszWqxM5Tgf2oiPO1wf2KBZQkW3t91t/RYD+TaahR+NOut7tzpS4/oflU7mJaIUkiP0kT6RzR2L6n5/OY8f1YR194sEAJ8x1Ol9h0Z/ogCYPgXM6HtM4DqILUAs9n4JHKn8QHVqix5v1fahtZ3p7pvq6hTPq0XvQ4t/XW1nbh
1vStUVL7Mkj/n1uRghmCtMSFzVMs2o7HPDh4LnzBboMYCNOw9ylOzGL72JnFqr55hrqGm1chmNB0oE+bdGnzfu0RZ92qU+7Hc38Qe4GqaCwYxuVXlRhyAmGQktbqdCjWPKGemL+q0KFKleehcqhjOqU2BsOo90mfuC31XW6FFlft6dlbaVTtqTCMDacVLWG3yGGgZ/OpjOP3MG
xFrFmSONgFgFUX0fBxPUA4p00KmWIn8EhdAgpxDUm+SIV4SheAQV1ERRErBJ4sCTW8YOrJwqh2GEakmhGpfn8t5dqIsMHJGaUQGwvcxQ2ImQo7xMgkaEgQ0GGggwFGcq/gaGsyUvosgHhOmAnPqlot2RDXmE4L9IibuRnaUqTyB3HEELoeAZRvBSUud4T6TvxZky/rp1IF6mY1
GMchY1iBuHwE8RAiKW8YyyaGFoxi+rrFsHDFi3phr2DJe11S/1wtjQnw1QNWZZ3MKfXm4toPPMSjhq5JcvStlq+ekbaAAL60QvGj99Chwj6yYuttJzTUFE7ID7wgoizFtCWbgQvSk4E8T8CBX8EMbPxAL/gaI4faZTHesY1gghaDVVT9XlmjMvfRQFxxiROfsg/QpeOqSxty0T
ENmxGRBa4gaop1hI9qF0uRp2zzOPr5Jz1zhdRFETDhe/Qv/lTKsGfcwKTdbskM3/8sBI0TMM82W39LwOISW7yzNjm8Kp/Phr9uLzqn16y2YSgz+M4nq/bZ5owtG0N+SlIxwHj+sQo2+q4bA0+OYgTkKv4r+uVi78ulAkflXqmnB9Mc5fDv1VUy4wqulEKpsa+Q1xuNo9xhwt6m
W1bVXOz/HmLALjf86Tk50lZd56U+vNkGw09T+ZRz5OiHuVAqfYh/Nnq2sVELVlpkEOruUOr6xxarXVoW+0206FN4/AOrZX82TyUP6uFO2ta98DurBnbse49u7OWu7O2zp21enfW5Ia6s3pcd977p7jcnQvs15StsP8XuLNqWlsBwivufO47mzvzzTP/GCYcgtVIaQNoATchMft
E8SJlT/BRn+ekpDmSfyT/SP75acxOTOazA2GJHYsv3wd/UBJmH+FhKlBzNZksVV75rEIogBNSbhF9K5u4UJpDWBWpaOAC33x34lJnVWC14cyN6Di5DgKvVDkgYUidpcrrKGBh4SaiFCb4vbZlOBpVt10/Jw+BXyMoGmtlr5IHGtWI8rZFyYHrB9HnPt/RckUeAAbk51IHUZF3u
AkS4uXlbP9L2567QulkpwHFLEN9ubl//a2IQXkPOJ1ZJNLLH+KK3sUpuaYk80IWZktTTkOcyP5w/4a9aH3uFwazHulwSClZW1w1fXxOaNwahWnUy+6cVqtHD7PJxKMMQJzyugxBT0JbPC8Wtz7RhN1rCTBgsbuqgT+3/iRuUh6sUMPtMhvVta2bIM2mFQYWRfhGQnEcRE5ePS/
G/2fkJrQ8gaxB6GHNALsLUxYi+VCLlRA2CpE5CzLhLCmrX5nR8sCuZkmlSHkklaayjKdYYXYh2GQSgRkPzHgcOuOBKI4ofjwU3xy+uZcjfCN87zOlpecArK8DYL0+pbVwQhoEwLZ+3JTWLl8X2C2lpRcpLXMrLPwF8KvrmvVuUlqY38X87mHzu8hFkYtuzEW1HakoOxPIRJGJN
[error] java.util.zip.ZipException: incorrect header check

Describe the solution you'd like
A clear and concise description of what you want to happen.
need decode method for payload data item.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

print("readfile")

import json

file1="payload-1674915807343-1.json"
file2="payload-1674915815120-2.json"

with open(file2, 'r') as f:
data = json.load(f)

Output: {'name': 'Bob', 'languages': ['English', 'French']}

dataitem = data["data"]
print("\n")
print("dmAppId: "+ data["dmAppId"])
print("\n")
print("counters ", data["counters"])
print("\n")

import base64

zipdata_decoded=base64.b64decode(dataitem)
print("\n===================")
print(zipdata_decoded)

import gzip, zlib

ba = bytearray(zipdata_decoded)
rt1=zlib.decompress(ba, 15+32)
rt2=zlib.decompress(bytes(ba), 15+32) #15 + 16
print("\n =================rt1")
print(rt1)

print("\n ============rt2")
print(rt2)