Copy problem megatools
darkistan opened this issue · comments
When I start copying, I get the following error.
The command is as follows megacopy --local /mnt/data/dump --remote /Root/SHADOW-SRV/HTZ
error.
ERROR: Upload failed for /Root/SHADOW-SRV/HTZ/vzdump-qemu-105-2021_09_28-23_15_1 8.vma.zst: API call 'p' failed: HTTP POST failed: CURL error: SSL connect error F
Tell me what is the reason for the error?
I confirm there is an error
megatools 1.11.0
megatools copy --reload --no-progress --local=/tmp/site --remote=/Root/backup/day_2021-10-14_1634158800
ERROR: Upload failed for /Root/backup/day_2021-10-14_1634158800/files.zip: API call 'p' failed: HTTP POST failed: CURL error: SSL connect error
Solved the problem by rewriting the script code. I added multiple repetitions of the command through megatools. I try to loop through the command until it succeeds.
My Bash function wrapper:
function run()
{
echo "------------------------------------------------------------" | LOG
OUTPUT=''
OUTPUT_ERROR=''
local cmd="$@"
echo "CMD: [${cmd}]" | LOG
local maxNumPut=10
# Несколько попыток выполнения команды, при EXIT_CODE > 0
for ((i=1; i <= $maxNumPut; i++))
do
echo "${FUNCNAME[0]}: $i попытка выполнения запроса" | LOG
#${cmd} >${OUT} 2>${OUT_ERROR}
eval "${cmd} >${OUT} 2>${OUT_ERROR}"
EXIT_CODE="$?"
echo "EXIT_CODE=[${EXIT_CODE}]" | LOG
OUTPUT=`cat ${OUT}`
echo "OUTPUT: [${OUTPUT}]" | LOG
# Если файл с ошибками не пустой, то считываем его в OUTPUT_ERROR
if [ -s "${OUT_ERROR}" ]; then
OUTPUT_ERROR=`cat ${OUT_ERROR}`
echo "OUTPUT_ERROR: [${OUTPUT_ERROR}]" | LOG
fi
# Успешная передача файла
if [ "${EXIT_CODE}" == "0" ]; then
return
fi
if [ $i == $maxNumPut ]; then
echo "${FUNCNAME[0]} | ERROR | Команда выполнилась с EXIT_CODE > 0 за $maxNumPut попытки!" | LOG
f_check_error
fi
done
}
run megatools put --no-progress --path="${DIR_REMOTE}" "'${FILE_LOCAL}'"
it's not just limited to copying. when using megadl I will now routinely see: API call 'g' failed: HTTP POST failed: CURL error: SSL connect error. but the maintainer brushed this error off earlier in #268
Решил проблему переписав код скрипта. Я добавил многократное повторение команды через мегаппараты. Я пытаюсь выполнить команду в цикле, пока она не завершится успешно.
function run() { echo "------------------------------------------------------------" | LOG OUTPUT='' OUTPUT_ERROR='' local cmd="$@" echo "CMD: [${cmd}]" | LOG local maxNumPut=10 # Несколько попыток выполнения команды, при EXIT_CODE > 0 for ((i=1; i <= $maxNumPut; i++)) do echo "${FUNCNAME[0]}: $i попытка выполнения запроса" | LOG #${cmd} >${OUT} 2>${OUT_ERROR} eval "${cmd} >${OUT} 2>${OUT_ERROR}" EXIT_CODE="$?" echo "EXIT_CODE=[${EXIT_CODE}]" | LOG OUTPUT=`cat ${OUT}` echo "OUTPUT: [${OUTPUT}]" | LOG # Если файл с ошибками не пустой, то считываем его в OUTPUT_ERROR if [ -s "${OUT_ERROR}" ]; then OUTPUT_ERROR=`cat ${OUT_ERROR}` echo "OUTPUT_ERROR: [${OUTPUT_ERROR}]" | LOG fi # Успешная передача файла if [ "${EXIT_CODE}" == "0" ]; then return fi if [ $i == $maxNumPut ]; then echo "${FUNCNAME[0]} | ERROR | Команда выполнилась с EXIT_CODE > 0 за $maxNumPut попытки!" | LOG f_check_error fi done }
run megatools put --no-progress --path="${DIR_REMOTE}" "'${FILE_LOCAL}'"
А где именно лежит файл данного скрипта ?
re-run your upload with --debug=api,http and look for the failure reason. this is what I got:
* Too old connection (1004 seconds), disconnect it
* Connection 0 seems to be dead!
* Closing connection 0
* Trying 66.203.125.11:443...
* Connected to g.api.mega.co.nz (66.203.125.11) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL re-using session ID
* error:1421C0F8:SSL routines:set_client_ciphersuite:unknown cipher returned
* Closing connection 1
ERROR: Upload failed for 'file.bin': API call 'p' failed: HTTP POST failed: CURL error: SSL connect error
so it looks like it is a MEGA/OS issue and not specific to megatools?
debug on a successful upload:
* Trying 66.203.125.12:443...
* Connected to g.api.mega.co.nz (66.203.125.12) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL re-using session ID
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=*.api.mega.co.nz
* start date: Dec 6 00:00:00 2020 GMT
* expire date: Dec 6 23:59:59 2021 GMT
* subjectAltName: host "g.api.mega.co.nz" matched cert's "*.api.mega.co.nz"
* issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* public key hash: sha256//0W38e765pAfPqS3DqSVOrPsC4MEOvRBaXQ7nY1AJ47E=
> POST /cs?id=1634916223&sid=KmBAVqNcLw_TmEecn12YfzNoMXVpU0lwZE40jPln6ZISXvlNz19UXv6Oxg HTTP/1.1
А где именно лежит файл данного скрипта ?
Это пример функции скрипта на bash. Я просто показал пример как можно решить данную проблему. Смысл в том, что я пытаюсь выполнить команду "megatools put ...." до тех пор, пока код возврата не будет нулём. Нолевой код возврата $? в bash означает успешное выполнение программы.
Я написал на bash функцию обёртку для всех выполняемых команд через megatools. Даже "megatools ls ..." через обёртку запускаю.
Помогло.
Maybe you can try disabling TLS session re-use in curl via:
curl_easy_setopt(curl, CURLOPT_SSL_SESSIONID_CACHE, 0L);
in http.c
I've added this change in the new builds. Maybe it will help.
thx
I've re-built the package, because last one had wrong pcre build configuration that caused some issues.
@megous both win32 and win64 builds of megatools-1.11.0-git-20211028 and megatools-1.11.0-git-20211029 fail to run with the following error:
Did the TLS session cache change help with uploads?
@megous I've had no curl errors since updating to 20211030-linux-x86_64.tar.gz (8/8 success rate on uploads)
just re-tested the previous experimental version and got the SSL error on first upload. fixed for me, at least.
Great. Thanks for confirming. :)
@megous FYI, from the build that fixed this (20211030) onwards, megatools (dl) cannot parse old-style mega (folder) links, for example this link (fails on both windows and linux, all builds):
./megatools dl 'https://mega.nz/#F!WwZyBZRL!1vXiBr7pJZLINpSRErBxvA' # 20211030, 20220401
ERROR: Can't open folder 'https://mega.nz/#F!WwZyBZRL!1vXiBr7pJZLINpSRErBxvA': Node not found:
Curiously, megatools built from corresponding source on ubuntu 20.04 doesn't exhibit this problem, so it seems this may be related to updated dependencies used in the published experimental builds.