Output format as CSV
jeffcouch21 opened this issue · comments
My use case is to run this in a cronjob every day, exporting the output to a database for long term capacity trend reporting at a namespace level.
Can you provide more details, like an example of output csv (column, value) ?
The 2 "tricky" points are :
- how to express aggregate resources
- how to express quantities provided in different units eg (MB + KiB)
One of the possibility is to have the same ouput as currently but:
- comma separated ;-)
- the resource column (the tree) express as a multicolumn
- quantities with units
So something like
> kubectl-view-allocations -r gpu
Resource Requested %Requested Limit %Limit Allocatable Free
nvidia.com/gpu 7 58% 7 58% 12 5
├─ node-gpu1 1 50% 1 50% 2 1
│ └─ xxxx-784dd998f4-zt9dh 1 1
├─ node-gpu2 0 0% 0 0% 2 2
├─ node-gpu3 0 0% 0 0% 2 2
├─ node-gpu4 1 50% 1 50% 2 1
│ └─ aaaa-1571819245-5ql82 1 1
├─ node-gpu5 2 100% 2 100% 2 0
│ ├─ bbbb-1571738839-dfkhn 1 1
│ └─ bbbb-1571738888-52c4w 1 1
└─ node-gpu6 2 100% 2 100% 2 0
├─ bbbb-1571738688-vlxng 1 1
└─ cccc-1571745684-7k6bn 1 1could become
> kubectl-view-allocations -r gpu -o csv
nvidia.com/gpu,,,7,58%,7,58%,12,5,
nvidia.com/gpu,node-gpu1,,1,50%,1,50%,2,1,
nvidia.com/gpu,node-gpu1,xxxx-784dd998f4-zt9dh,1,,1,,,,
nvidia.com/gpu,node-gpu2,,0,0%,0,0%,2,2,
nvidia.com/gpu,node-gpu3,,0,0%,0,0%,2,2,
nvidia.com/gpu,node-gpu4,,1,50%,1,50%,2,1,
nvidia.com/gpu,node-gpu4,aaaa-1571819245-5ql82,1,,1,,,,
nvidia.com/gpu,node-gpu5,,2,100%,2,100%,2,0,
nvidia.com/gpu,node-gpu5,bbbb-1571738839-dfkhn,1,,1,,,,
nvidia.com/gpu,node-gpu5,bbbb-1571738888-52c4w,1,,1,,,,
nvidia.com/gpu,node-gpu6,,2,100%,2,100%,2,0,
nvidia.com/gpu,node-gpu6,bbbb-1571738688-vlxng,1,,1,,,,
nvidia.com/gpu,node-gpu6,cccc-1571745684-7k6,1,,1,,,,- I can work with what you have as an example. I think even with quantities + units, its def workable. But I would think there needs to be some kind of field to denote the aggregation
- If I were to be picky, i would eliminate the aggregation and output the requested "group" (namespace/pod)
- Include a system timestamp as a column
Date,Resource,Namespace,Requested,Limit
2020-12-06 02:34:56,cpu,ns1,1250.00,2400
2020-12-06 02:34:56,cpu,ns2,2700.00,2700
2020-12-06 02:34:56,cpu,ns3,900.00,9
2020-12-06 02:34:56,cpu,ns4,10.00,10
2020-12-06 02:34:56,cpu,ns5,2570.00,0
I will tell you where I am headed next with this data set. We are starting to roll out some quotas and need to report regularly on how much quota has been handed out in the cluster (aggregated) , what was handed out in each namespace (quota value), and how much is consumed of the quota (allocated). I mocked up this CSV to show a start. It includes those namespaces that do not include a resource quota object as well.
Date,Resource,Cluster,Namespace,NamespaceResourceQuota,NamespaceResourceQuotaRequest,NamespaceResourceQuotaLimit,Requested,%Requested,Limit,%Limit
2020-12-06 02:34:56,cpu,CL1,NS1,NSQ1,2500,5000,1250,0.50,2400,0.48
2020-12-06 02:34:56,cpu,CL1,NS2,NSQ2,5000,5000,2700,0.54,2700,0.54
2020-12-06 02:34:56,cpu,CL1,NS3,null,null,null,900,null,9,null
2020-12-06 02:34:56,cpu,CL1,NS4,null,null,null,10,null,10,null
2020-12-06 02:34:56,cpu,CL1,NS5,NSQ4,5000,5000,2570,0.51,100,0.02
I will definitely open another "issue" for adding a resource quota, potentially as another way to group objects(?). IMHO anyone running a self-service multi-tenant cluster(s) needs to be thinking about a resource quota solution. The current plan is to allocate/de-allocate quota to namespaces and scale the cluster to meet the quota demand.
I'm releasing a version 0.8.2 with export csv. I provide 2 output samples into the README.
- timestamp follow RFC 3339 and ISO 8601
- a Kind column is provided to help filter line, eg if you want aggregated
- support the group-by arguments to change the way to aggregate and the displayed lines
@davidB this works great! Going to roll with it and let you know if I find anything that seems off.