heketi: when create pvc for statefulset (prometheus-k8s) , the volume has been created .but after few senconds, the volume was been deleted
Phoaster-wry opened this issue · comments
I created a storage volume on the interface, but found in the heketi log, and it was deleted immediately after creation, I really don't understand why.
logs as follow.##########################
Setting up heketi database
File: /var/lib/heketi/heketi.db
Size: 1048576 Blocks: 1792 IO Block: 131072 regular file
Device: cbh/203d Inode: 12951443882676486076 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2022-06-02 01:41:36.256000000 +0000
Modify: 2022-12-16 10:11:44.869237354 +0000
Change: 2022-12-16 10:11:44.881237354 +0000
Birth: -
Heketi v7.0.0
[heketi] INFO 2022/12/16 10:24:38 Loaded ssh executor
[heketi] INFO 2022/12/16 10:24:38 GlusterFS Application Loaded
[heketi] INFO 2022/12/16 10:24:38 Started Node Health Cache Monitor
Authorization loaded
Listening on port 8080
[heketi] INFO 2022/12/16 10:24:48 Starting Node Health Status refresh
[cmdexec] INFO 2022/12/16 10:24:48 Check Glusterd service status in node 10.227.105.5
[cmdexec] DEBUG 2022/12/16 10:24:48 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'systemctl status glusterd'
Result: ● glusterd.service - LSB: GlusterFS server
Loaded: loaded (/etc/init.d/glusterfs-server; static; vendor preset: enabled)
Active: active (running) since 五 2022-12-16 18:23:17 CST; 1min 31s ago
Docs: man:systemd-sysv-generator(8)
Process: 47872 ExecStop=/etc/init.d/glusterfs-server stop (code=exited, status=0/SUCCESS)
Process: 47907 ExecStart=/etc/init.d/glusterfs-server start (code=exited, status=0/SUCCESS)
Tasks: 219
Memory: 1.4G
CPU: 5.672s
CGroup: /system.slice/glusterd.service
├─16960 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id heketidbstorage.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_2799aa1ddb3bdeb772bdb76e8b1007de-brick -p /var/lib/glusterd/vols/heketidbstorage/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_2799aa1ddb3bdeb772bdb76e8b1007de-brick.pid -S /var/run/gluster/e96aaa36ea6c89c74bc43d452073eab1.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_2799aa1ddb3bdeb772bdb76e8b1007de/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_2799aa1ddb3bdeb772bdb76e8b1007de-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49152 --xlator-option heketidbstorage-server.listen-port=49152
├─16996 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_1f5b051f212714fbf487101ab14b78a9.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_3710161a0a0935f20986a80164201534-brick -p /var/lib/glusterd/vols/vol_1f5b051f212714fbf487101ab14b78a9/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_3710161a0a0935f20986a80164201534-brick.pid -S /var/run/gluster/7253700f2269a4303a93d84b486f9b2f.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_3710161a0a0935f20986a80164201534/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_3710161a0a0935f20986a80164201534-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49160 --xlator-option vol_1f5b051f212714fbf487101ab14b78a9-server.listen-port=49160
├─17007 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_44efcd7c807426951a40ed8c95128ef5.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_bb698ac7648291ccc22d603b48d20ab5-brick -p /var/lib/glusterd/vols/vol_44efcd7c807426951a40ed8c95128ef5/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_bb698ac7648291ccc22d603b48d20ab5-brick.pid -S /var/run/gluster/2d1fc027c22324c24c3c843047ce8bea.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bb698ac7648291ccc22d603b48d20ab5/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_bb698ac7648291ccc22d603b48d20ab5-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49162 --xlator-option vol_44efcd7c807426951a40ed8c95128ef5-server.listen-port=49162
├─17022 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_477ebd978f8d9c44ef97492aebe88102.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_71c176ba22ebcece184f98b4b2315f4f-brick -p /var/lib/glusterd/vols/vol_477ebd978f8d9c44ef97492aebe88102/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_71c176ba22ebcece184f98b4b2315f4f-brick.pid -S /var/run/gluster/8ef907e3ff8318eb6f45753f1137544a.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_71c176ba22ebcece184f98b4b2315f4f/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_71c176ba22ebcece184f98b4b2315f4f-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49154 --xlator-option vol_477ebd978f8d9c44ef97492aebe88102-server.listen-port=49154
├─17034 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_5caa41e58488f4b0d058e81edb930290.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_67c4a3350d55d2c3dc0fdde4e8e34488-brick -p /var/lib/glusterd/vols/vol_5caa41e58488f4b0d058e81edb930290/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_67c4a3350d55d2c3dc0fdde4e8e34488-brick.pid -S /var/run/gluster/d207762fcc40a241979cd83fa74071f8.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_67c4a3350d55d2c3dc0fdde4e8e34488/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_67c4a3350d55d2c3dc0fdde4e8e34488-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49164 --xlator-option vol_5caa41e58488f4b0d058e81edb930290-server.listen-port=49164
├─17048 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_6900c307ae6a86bb7dac4ba54ec04237.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_266eb33621acd9dd89b35712e693a40b-brick -p /var/lib/glusterd/vols/vol_6900c307ae6a86bb7dac4ba54ec04237/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_266eb33621acd9dd89b35712e693a40b-brick.pid -S /var/run/gluster/3a0337c067864ac6d6b1a578721fa203.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_266eb33621acd9dd89b35712e693a40b/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_266eb33621acd9dd89b35712e693a40b-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49159 --xlator-option vol_6900c307ae6a86bb7dac4ba54ec04237-server.listen-port=49159
├─17062 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_6987924342694c8a11d2427290473d53.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_52e001743fc0af02ed6206f033d8e127-brick -p /var/lib/glusterd/vols/vol_6987924342694c8a11d2427290473d53/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_52e001743fc0af02ed6206f033d8e127-brick.pid -S /var/run/gluster/deaef404f67c127543ad08de071a94dd.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_52e001743fc0af02ed6206f033d8e127/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_52e001743fc0af02ed6206f033d8e127-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49161 --xlator-option vol_6987924342694c8a11d2427290473d53-server.listen-port=49161
├─17076 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_74a200f3ba157b0037072e01eb75716b.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_d4ebf3f312a940f8862d454ecb019dc1-brick -p /var/lib/glusterd/vols/vol_74a200f3ba157b0037072e01eb75716b/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_d4ebf3f312a940f8862d454ecb019dc1-brick.pid -S /var/run/gluster/9e91d1e49969d5911c40be2e2abcc764.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_d4ebf3f312a940f8862d454ecb019dc1/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_d4ebf3f312a940f8862d454ecb019dc1-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49155 --xlator-option vol_74a200f3ba157b0037072e01eb75716b-server.listen-port=49155
├─17097 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_c3c5b7cc41f6253227de65dd345765fa.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_4a3ccdf10aa7d5b62f26689621f47053-brick -p /var/lib/glusterd/vols/vol_c3c5b7cc41f6253227de65dd345765fa/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_4a3ccdf10aa7d5b62f26689621f47053-brick.pid -S /var/run/gluster/a42eae0dd46134a110c3bddba2102338.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_4a3ccdf10aa7d5b62f26689621f47053/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_4a3ccdf10aa7d5b62f26689621f47053-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49168 --xlator-option vol_c3c5b7cc41f6253227de65dd345765fa-server.listen-port=49168
├─17106 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_8b39c0260daade57c3a48c2a1fcfa764.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_ddae7b32f38a5c0eaedc823f9b9ca6cd-brick -p /var/lib/glusterd/vols/vol_8b39c0260daade57c3a48c2a1fcfa764/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_ddae7b32f38a5c0eaedc823f9b9ca6cd-brick.pid -S /var/run/gluster/783682ba6215e408632144ffe958948f.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_ddae7b32f38a5c0eaedc823f9b9ca6cd/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_ddae7b32f38a5c0eaedc823f9b9ca6cd-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49163 --xlator-option vol_8b39c0260daade57c3a48c2a1fcfa764-server.listen-port=49163
├─17112 /usr/sbin/glusterfsd -s 10.227.105.5 --volfile-id vol_d4142a264c486e375bbe62ad4058550e.10.227.105.5.var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_96565219dec77b11d1960259b5bf2d0d-brick -p /var/lib/glusterd/vols/vol_d4142a264c486e375bbe62ad4058550e/run/10.227.105.5-var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_96565219dec77b11d1960259b5bf2d0d-brick.pid -S /var/run/gluster/e25681402331d3818eecd583f6358d07.socket --brick-name /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_96565219dec77b11d1960259b5bf2d0d/brick -l /var/log/glusterfs/bricks/var-lib-heketi-mounts-vg_c16e46324ca83678af2bced832776133-brick_96565219dec77b11d1960259b5bf2d0d-brick.log --xlator-option *-posix.glusterd-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b --brick-port 49156 --xlator-option vol_d4142a264c486e375bbe62ad4058550e-server.listen-port=49156
├─47925 /usr/sbin/glusterd -p /var/run/glusterd.pid
└─52547 /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/lib/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/55ea83d47c2941486ba1d938a3370990.socket --xlator-option replicate.node-uuid=d929bd50-a484-43cb-ad3f-a92f687a887b
.............................
[asynchttp] INFO 2022/12/16 10:29:41 asynchttp.go:288: Started job ebc6c8f30beedb32873bca1b67443508
[heketi] INFO 2022/12/16 10:29:41 Started async operation: Create Volume
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 196.18µs
[heketi] INFO 2022/12/16 10:29:41 Creating brick ceed53844f012ec9eeffe5ed75830d7a
[heketi] INFO 2022/12/16 10:29:41 Creating brick eacc3aabe84e3581c2406cb440de6ae7
[heketi] INFO 2022/12/16 10:29:41 Creating brick 5e4e6096dc66de0978b79603ba9a5e80
[heketi] INFO 2022/12/16 10:29:41 Allocating brick set #0
[asynchttp] INFO 2022/12/16 10:29:41 asynchttp.go:288: Started job 2ca6c9ae5e0ccd3493e30ed30c90dfe2
[heketi] INFO 2022/12/16 10:29:41 Started async operation: Create Volume
[negroni] Completed 202 Accepted in 46.67716ms
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 1.5306ms
[heketi] INFO 2022/12/16 10:29:41 Creating brick d2474a9af71aa4b389366d1d0861199f
[heketi] INFO 2022/12/16 10:29:41 Creating brick 044333fc742f609b350b82fdc916c7f7
[heketi] INFO 2022/12/16 10:29:41 Creating brick bc8526d9305d929b1bebf3fc913da93d
[cmdexec] DEBUG 2022/12/16 10:29:41 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mkdir -p /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:41 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mkdir -p /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:41 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mkdir -p /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a'
Result:
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 208.24µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 244.94µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 152.12µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 206.16µs
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvcreate --autobackup=n --poolmetadatasize 106496K --chunksize 256K --size 20971520K --thin vg_99d0d4a9da86ba547d7322d05958c5e8/tp_ceed53844f012ec9eeffe5ed75830d7a --virtualsize 20971520K --name brick_ceed53844f012ec9eeffe5ed75830d7a'
Result: Logical volume "brick_ceed53844f012ec9eeffe5ed75830d7a" created.
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvcreate --autobackup=n --poolmetadatasize 106496K --chunksize 256K --size 20971520K --thin vg_c16e46324ca83678af2bced832776133/tp_eacc3aabe84e3581c2406cb440de6ae7 --virtualsize 20971520K --name brick_eacc3aabe84e3581c2406cb440de6ae7'
Result: Logical volume "brick_eacc3aabe84e3581c2406cb440de6ae7" created.
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvcreate --autobackup=n --poolmetadatasize 106496K --chunksize 256K --size 20971520K --thin vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_5e4e6096dc66de0978b79603ba9a5e80 --virtualsize 20971520K --name brick_5e4e6096dc66de0978b79603ba9a5e80'
Result: Logical volume "brick_5e4e6096dc66de0978b79603ba9a5e80" created.
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 170.88µs
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a'
Result: meta-data=/dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a isize=512 agcount=16, agsize=327616 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=5241856, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =无 extsz=4096 blocks=0, rtextents=0
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'awk "BEGIN {print "/dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a xfs rw,inode64,noatime,nouuid 1 2" >> "/etc/fstab"}"'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7'
Result: meta-data=/dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7 isize=512 agcount=16, agsize=327616 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=5241856, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =无 extsz=4096 blocks=0, rtextents=0
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'awk "BEGIN {print "/dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7 /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7 xfs rw,inode64,noatime,nouuid 1 2" >> "/etc/fstab"}"'
Result:
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 192.48µs
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80'
Result: meta-data=/dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80 isize=512 agcount=16, agsize=327616 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=5241856, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =无 extsz=4096 blocks=0, rtextents=0
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'awk "BEGIN {print "/dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80 xfs rw,inode64,noatime,nouuid 1 2" >> "/etc/fstab"}"'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mount -o rw,inode64,noatime,nouuid /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:42 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mkdir /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mount -o rw,inode64,noatime,nouuid /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7 /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'chown :2003 /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mkdir /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'chmod 2775 /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'chown :2003 /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mount -o rw,inode64,noatime,nouuid /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'chmod 2775 /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mkdir /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'chown :2003 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'chmod 2775 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80/brick'
Result:
[cmdexec] INFO 2022/12/16 10:29:43 Creating volume vol_3dd3d2262365db7196ac7f084673f78d replica 3
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mkdir -p /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mkdir -p /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d'
Result:
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 141.86µs
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mkdir -p /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7'
Result:
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 153µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 219.28µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 198.08µs
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvcreate --autobackup=n --poolmetadatasize 106496K --chunksize 256K --size 20971520K --thin vg_99d0d4a9da86ba547d7322d05958c5e8/tp_d2474a9af71aa4b389366d1d0861199f --virtualsize 20971520K --name brick_d2474a9af71aa4b389366d1d0861199f'
Result: Logical volume "brick_d2474a9af71aa4b389366d1d0861199f" created.
[cmdexec] DEBUG 2022/12/16 10:29:43 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvcreate --autobackup=n --poolmetadatasize 106496K --chunksize 256K --size 20971520K --thin vg_c16e46324ca83678af2bced832776133/tp_bc8526d9305d929b1bebf3fc913da93d --virtualsize 20971520K --name brick_bc8526d9305d929b1bebf3fc913da93d'
Result: Logical volume "brick_bc8526d9305d929b1bebf3fc913da93d" created.
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 197.14µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 159µs
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvcreate --autobackup=n --poolmetadatasize 106496K --chunksize 256K --size 20971520K --thin vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_044333fc742f609b350b82fdc916c7f7 --virtualsize 20971520K --name brick_044333fc742f609b350b82fdc916c7f7'
Result: Logical volume "brick_044333fc742f609b350b82fdc916c7f7" created.
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f'
Result: meta-data=/dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f isize=512 agcount=16, agsize=327616 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=5241856, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =无 extsz=4096 blocks=0, rtextents=0
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'awk "BEGIN {print "/dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f xfs rw,inode64,noatime,nouuid 1 2" >> "/etc/fstab"}"'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d'
Result: meta-data=/dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d isize=512 agcount=16, agsize=327616 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=5241856, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =无 extsz=4096 blocks=0, rtextents=0
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'awk "BEGIN {print "/dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d xfs rw,inode64,noatime,nouuid 1 2" >> "/etc/fstab"}"'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mount -o rw,inode64,noatime,nouuid /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mkdir /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'chown :2004 /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mount -o rw,inode64,noatime,nouuid /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mkdir /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'chmod 2775 /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'chown :2004 /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'chmod 2775 /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7'
Result: meta-data=/dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7 isize=512 agcount=16, agsize=327616 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=5241856, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =无 extsz=4096 blocks=0, rtextents=0
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'awk "BEGIN {print "/dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7 xfs rw,inode64,noatime,nouuid 1 2" >> "/etc/fstab"}"'
Result:
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 179.68µs
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mount -o rw,inode64,noatime,nouuid /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mkdir /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7/brick'
Result:
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 170.2µs
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'chown :2004 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7/brick'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:44 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'chmod 2775 /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7/brick'
Result:
[cmdexec] INFO 2022/12/16 10:29:44 Creating volume vol_824ae4d04a18bc685695d9c786ef764c replica 3
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 202.28µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 235.72µs
[cmdexec] DEBUG 2022/12/16 10:29:45 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'gluster --mode=script volume create vol_3dd3d2262365db7196ac7f084673f78d replica 3 10.227.105.5:/var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7/brick 10.227.105.3:/var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a/brick 10.227.105.4:/var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80/brick '
Result: volume create: vol_3dd3d2262365db7196ac7f084673f78d: success: please start the volume to access data
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 190.14µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 224.92µs
[cmdexec] DEBUG 2022/12/16 10:29:45 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'gluster --mode=script volume create vol_824ae4d04a18bc685695d9c786ef764c replica 3 10.227.105.3:/var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f/brick 10.227.105.4:/var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7/brick 10.227.105.5:/var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d/brick '
Result: volume create: vol_824ae4d04a18bc685695d9c786ef764c: success: please start the volume to access data
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 181µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 191.24µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 167.36µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 156.94µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 187.9µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 167.36µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 171.86µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 162.04µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 174.34µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 164.92µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 158.78µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 178.9µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 196.28µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 188.76µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 186.94µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 159.68µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 175.58µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 188.22µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 195.34µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 196.32µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 173µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 178.12µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 171.36µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 215.38µs
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 200 OK in 196.2µs
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 200 OK in 168.52µs
[cmdexec] DEBUG 2022/12/16 10:29:50 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'gluster --mode=script volume start vol_3dd3d2262365db7196ac7f084673f78d'
Result: volume start: vol_3dd3d2262365db7196ac7f084673f78d: success
[heketi] INFO 2022/12/16 10:29:50 Create Volume succeeded
[asynchttp] INFO 2022/12/16 10:29:50 asynchttp.go:292: Completed job ebc6c8f30beedb32873bca1b67443508 in 8.995834673s
[cmdexec] DEBUG 2022/12/16 10:29:50 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'gluster --mode=script volume start vol_824ae4d04a18bc685695d9c786ef764c'
Result: volume start: vol_824ae4d04a18bc685695d9c786ef764c: success
[heketi] INFO 2022/12/16 10:29:50 Create Volume succeeded
[asynchttp] INFO 2022/12/16 10:29:50 asynchttp.go:292: Completed job 2ca6c9ae5e0ccd3493e30ed30c90dfe2 in 8.997298493s
[negroni] Started GET /queue/ebc6c8f30beedb32873bca1b67443508
[negroni] Completed 303 See Other in 179.98µs
[negroni] Started GET /volumes/3dd3d2262365db7196ac7f084673f78d
[negroni] Completed 200 OK in 4.0176ms
[negroni] Started GET /clusters/29536b7df77aaf0f4dc00065fd03850a
[negroni] Completed 200 OK in 496.64µs
[negroni] Started GET /nodes/5bffc27d3128a53c9ece4b0ca1e45a92
[negroni] Completed 200 OK in 2.40622ms
[negroni] Started GET /nodes/9dfa0912d3d08fcc3e5b8ea0df4d257a
[negroni] Completed 200 OK in 2.10884ms
[negroni] Started GET /nodes/f1551418b582927754206664442d2cac
[negroni] Completed 200 OK in 2.15082ms
[negroni] Started GET /queue/2ca6c9ae5e0ccd3493e30ed30c90dfe2
[negroni] Completed 303 See Other in 242.24µs
[negroni] Started GET /volumes/824ae4d04a18bc685695d9c786ef764c
[negroni] Completed 200 OK in 1.32014ms
[negroni] Started GET /clusters/29536b7df77aaf0f4dc00065fd03850a
[negroni] Completed 200 OK in 349.58µs
[negroni] Started GET /nodes/5bffc27d3128a53c9ece4b0ca1e45a92
[negroni] Completed 200 OK in 2.13636ms
[negroni] Started GET /nodes/9dfa0912d3d08fcc3e5b8ea0df4d257a
[negroni] Completed 200 OK in 2.26342ms
[negroni] Started GET /nodes/f1551418b582927754206664442d2cac
[negroni] Completed 200 OK in 2.30088ms
[negroni] Started DELETE /volumes/3dd3d2262365db7196ac7f084673f78d
[negroni] Started DELETE /volumes/824ae4d04a18bc685695d9c786ef764c
[negroni] Completed 202 Accepted in 35.36378ms
[asynchttp] INFO 2022/12/16 10:29:51 asynchttp.go:288: Started job c325227a2276e4046af74d3a6851a41f
[heketi] INFO 2022/12/16 10:29:51 Started async operation: Delete Volume
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 200.7µs
[negroni] Completed 202 Accepted in 43.1182ms
[asynchttp] INFO 2022/12/16 10:29:51 asynchttp.go:288: Started job 1fd7e744903dbf89a8118ae58429c50f
[heketi] INFO 2022/12/16 10:29:51 Started async operation: Delete Volume
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 232.94µs
[cmdexec] DEBUG 2022/12/16 10:29:51 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'gluster --mode=script snapshot list vol_3dd3d2262365db7196ac7f084673f78d --xml'
Result:
0
0
0
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 162.36µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 153.28µs
[cmdexec] DEBUG 2022/12/16 10:29:51 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'gluster --mode=script snapshot list vol_824ae4d04a18bc685695d9c786ef764c --xml'
Result:
0
0
0
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 185.88µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 189.42µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 164.9µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 195.32µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 209.04µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 2.7012ms
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 187.8µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 170.18µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 188.56µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 163.66µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 161.04µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 154.7µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 195.02µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 186.56µs
[cmdexec] DEBUG 2022/12/16 10:29:54 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'gluster --mode=script volume stop vol_3dd3d2262365db7196ac7f084673f78d force'
Result: volume stop: vol_3dd3d2262365db7196ac7f084673f78d: success
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 173.9µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 157.7µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 423.5µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 176.06µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 153.64µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 174.34µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 168.88µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 145.76µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 190.9µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 153.18µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 177.74µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 160.36µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 173.78µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 239.1µs
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'gluster --mode=script volume stop vol_824ae4d04a18bc685695d9c786ef764c force'
Result: volume stop: vol_824ae4d04a18bc685695d9c786ef764c: success
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 200.84µs
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'gluster --mode=script volume delete vol_3dd3d2262365db7196ac7f084673f78d'
Result: volume delete: vol_3dd3d2262365db7196ac7f084673f78d: success
[heketi] INFO 2022/12/16 10:29:57 Deleting brick ceed53844f012ec9eeffe5ed75830d7a
[heketi] INFO 2022/12/16 10:29:57 Deleting brick 5e4e6096dc66de0978b79603ba9a5e80
[heketi] INFO 2022/12/16 10:29:57 Deleting brick eacc3aabe84e3581c2406cb440de6ae7
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 179µs
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mount | grep -w /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7 | cut -d" " -f1'
Result: /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mount | grep -w /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a | cut -d" " -f1'
Result: /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'gluster --mode=script volume delete vol_824ae4d04a18bc685695d9c786ef764c'
Result: volume delete: vol_824ae4d04a18bc685695d9c786ef764c: success
[heketi] INFO 2022/12/16 10:29:57 Deleting brick bc8526d9305d929b1bebf3fc913da93d
[heketi] INFO 2022/12/16 10:29:57 Deleting brick 044333fc742f609b350b82fdc916c7f7
[heketi] INFO 2022/12/16 10:29:57 Deleting brick d2474a9af71aa4b389366d1d0861199f
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 231.3µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 176.4µs
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvs --noheadings --separator=/ -ovg_name,pool_lv /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7
'
Result: vg_c16e46324ca83678af2bced832776133/tp_eacc3aabe84e3581c2406cb440de6ae7
[cmdexec] DEBUG 2022/12/16 10:29:57 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvs --noheadings --separator=/ -ovg_name,pool_lv /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a
'
Result: vg_99d0d4a9da86ba547d7322d05958c5e8/tp_ceed53844f012ec9eeffe5ed75830d7a
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mount | grep -w /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80 | cut -d" " -f1'
Result: /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'mount | grep -w /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d | cut -d" " -f1'
Result: /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'mount | grep -w /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f | cut -d" " -f1'
Result: /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'mount | grep -w /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7 | cut -d" " -f1'
Result: /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 153.18µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 158.34µs
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvs --noheadings --separator=/ -ovg_name,pool_lv /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80
'
Result: vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_5e4e6096dc66de0978b79603ba9a5e80
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'umount /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'umount /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a'
Result:
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 160µs
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvs --noheadings --separator=/ -ovg_name,pool_lv /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d
'
Result: vg_c16e46324ca83678af2bced832776133/tp_bc8526d9305d929b1bebf3fc913da93d
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 170.22µs
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvs --noheadings --separator=/ -ovg_name,pool_lv /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7
'
Result: vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_044333fc742f609b350b82fdc916c7f7
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvs --noheadings --separator=/ -ovg_name,pool_lv /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f
'
Result: vg_99d0d4a9da86ba547d7322d05958c5e8/tp_d2474a9af71aa4b389366d1d0861199f
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'sed -i.save "/brick_eacc3aabe84e3581c2406cb440de6ae7/d" /etc/fstab'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:58 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'sed -i.save "/brick_ceed53844f012ec9eeffe5ed75830d7a/d" /etc/fstab'
Result:
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 219.64µs
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'umount /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80'
Result:
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 176.14µs
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'umount /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'umount /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'umount /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7'
Result:
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 240.54µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 168.5µs
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'sed -i.save "/brick_5e4e6096dc66de0978b79603ba9a5e80/d" /etc/fstab'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvremove --autobackup=n -f /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_eacc3aabe84e3581c2406cb440de6ae7
'
Result: Logical volume "brick_eacc3aabe84e3581c2406cb440de6ae7" successfully removed
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvremove --autobackup=n -f /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_ceed53844f012ec9eeffe5ed75830d7a
'
Result: Logical volume "brick_ceed53844f012ec9eeffe5ed75830d7a" successfully removed
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'sed -i.save "/brick_044333fc742f609b350b82fdc916c7f7/d" /etc/fstab'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'sed -i.save "/brick_bc8526d9305d929b1bebf3fc913da93d/d" /etc/fstab'
Result:
[cmdexec] DEBUG 2022/12/16 10:29:59 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'sed -i.save "/brick_d2474a9af71aa4b389366d1d0861199f/d" /etc/fstab'
Result:
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 139.44µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 154.5µs
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvs --noheadings --options=thin_count vg_c16e46324ca83678af2bced832776133/tp_eacc3aabe84e3581c2406cb440de6ae7
'
Result: 0
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvs --noheadings --options=thin_count vg_99d0d4a9da86ba547d7322d05958c5e8/tp_ceed53844f012ec9eeffe5ed75830d7a
'
Result: 0
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvremove --autobackup=n -f /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_5e4e6096dc66de0978b79603ba9a5e80
'
Result: Logical volume "brick_5e4e6096dc66de0978b79603ba9a5e80" successfully removed
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 161.9µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 164.5µs
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvremove --autobackup=n -f /dev/mapper/vg_99d0d4a9da86ba547d7322d05958c5e8-brick_d2474a9af71aa4b389366d1d0861199f
'
Result: Logical volume "brick_d2474a9af71aa4b389366d1d0861199f" successfully removed
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvremove --autobackup=n -f /dev/mapper/vg_c16e46324ca83678af2bced832776133-brick_bc8526d9305d929b1bebf3fc913da93d
'
Result: Logical volume "brick_bc8526d9305d929b1bebf3fc913da93d" successfully removed
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvremove --autobackup=n -f /dev/mapper/vg_558d9dca1f7338e9fbaf6c9ddd03111d-brick_044333fc742f609b350b82fdc916c7f7
'
Result: Logical volume "brick_044333fc742f609b350b82fdc916c7f7" successfully removed
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 159.64µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 197.22µs
[cmdexec] DEBUG 2022/12/16 10:30:00 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvs --noheadings --options=thin_count vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_5e4e6096dc66de0978b79603ba9a5e80
'
Result: 0
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 143.2µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 168.64µs
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvs --noheadings --options=thin_count vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_044333fc742f609b350b82fdc916c7f7
'
Result: 0
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvremove --autobackup=n -f vg_99d0d4a9da86ba547d7322d05958c5e8/tp_ceed53844f012ec9eeffe5ed75830d7a
'
Result: Logical volume "tp_ceed53844f012ec9eeffe5ed75830d7a" successfully removed
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvremove --autobackup=n -f vg_c16e46324ca83678af2bced832776133/tp_eacc3aabe84e3581c2406cb440de6ae7
'
Result: Logical volume "tp_eacc3aabe84e3581c2406cb440de6ae7" successfully removed
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvs --noheadings --options=thin_count vg_99d0d4a9da86ba547d7322d05958c5e8/tp_d2474a9af71aa4b389366d1d0861199f
'
Result: 0
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvs --noheadings --options=thin_count vg_c16e46324ca83678af2bced832776133/tp_bc8526d9305d929b1bebf3fc913da93d
'
Result: 0
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 142.26µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 235.16µs
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'rmdir /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_ceed53844f012ec9eeffe5ed75830d7a'
Result:
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'rmdir /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_eacc3aabe84e3581c2406cb440de6ae7'
Result:
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 161.3µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 144.34µs
[cmdexec] DEBUG 2022/12/16 10:30:01 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvremove --autobackup=n -f vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_5e4e6096dc66de0978b79603ba9a5e80
'
Result: Logical volume "tp_5e4e6096dc66de0978b79603ba9a5e80" successfully removed
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 185.86µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 158.56µs
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 181.68µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 160.44µs
[cmdexec] DEBUG 2022/12/16 10:30:02 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'lvremove --autobackup=n -f vg_c16e46324ca83678af2bced832776133/tp_bc8526d9305d929b1bebf3fc913da93d
'
Result: Logical volume "tp_bc8526d9305d929b1bebf3fc913da93d" successfully removed
[cmdexec] DEBUG 2022/12/16 10:30:02 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'lvremove --autobackup=n -f vg_99d0d4a9da86ba547d7322d05958c5e8/tp_d2474a9af71aa4b389366d1d0861199f
'
Result: Logical volume "tp_d2474a9af71aa4b389366d1d0861199f" successfully removed
[cmdexec] DEBUG 2022/12/16 10:30:02 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.3:22 Command: /bin/bash -c 'rmdir /var/lib/heketi/mounts/vg_99d0d4a9da86ba547d7322d05958c5e8/brick_d2474a9af71aa4b389366d1d0861199f'
Result:
[cmdexec] DEBUG 2022/12/16 10:30:02 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.5:22 Command: /bin/bash -c 'rmdir /var/lib/heketi/mounts/vg_c16e46324ca83678af2bced832776133/brick_bc8526d9305d929b1bebf3fc913da93d'
Result:
[cmdexec] DEBUG 2022/12/16 10:30:03 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'lvremove --autobackup=n -f vg_558d9dca1f7338e9fbaf6c9ddd03111d/tp_044333fc742f609b350b82fdc916c7f7
'
Result: Logical volume "tp_044333fc742f609b350b82fdc916c7f7" successfully removed
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 200 OK in 151.76µs
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 132.96µs
[cmdexec] DEBUG 2022/12/16 10:30:03 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'rmdir /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_5e4e6096dc66de0978b79603ba9a5e80'
Result:
[heketi] INFO 2022/12/16 10:30:03 Delete Volume succeeded
[asynchttp] INFO 2022/12/16 10:30:03 asynchttp.go:292: Completed job c325227a2276e4046af74d3a6851a41f in 12.272776123s
[negroni] Started GET /queue/c325227a2276e4046af74d3a6851a41f
[negroni] Completed 204 No Content in 153.3µs
[cmdexec] DEBUG 2022/12/16 10:30:03 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.227.105.4:22 Command: /bin/bash -c 'rmdir /var/lib/heketi/mounts/vg_558d9dca1f7338e9fbaf6c9ddd03111d/brick_044333fc742f609b350b82fdc916c7f7'
Result:
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 200 OK in 130.36µs
[heketi] INFO 2022/12/16 10:30:03 Delete Volume succeeded
[asynchttp] INFO 2022/12/16 10:30:03 asynchttp.go:292: Completed job 1fd7e744903dbf89a8118ae58429c50f in 12.454743103s
[negroni] Started GET /queue/1fd7e744903dbf89a8118ae58429c50f
[negroni] Completed 204 No Content in 172.88µs
As noted for a long time in the project readme:
The Heketi project is now in deep maintenance status. This means that only critical bugs or security defects are being considered for inclusion by the project maintenance team. Expect the project to be archived soon. Expect slow replies to issues.
The only reason heketi project has not been archived yet is that there remain a few commercial users and in the unlikely chance those customers hit a critical bug and need a fix we've left the repo "live".
Please consider this a suggestion to migrate away from heketi as soon as you can. We have no bandwith any more to help with community issues. I hope you understand.