Reducing Time for _changes Request in CouchDB

Question

Reducing Time for _changes Request in CouchDB

homu-konamilk opened this issue 5 months ago · comments

Thank you for developing and maintaining the Obsidian LiveSync plugin; it's been incredibly useful for syncing notes across my devices. However, I've encountered a significant issue related to request timeouts that frequently disrupts the synchronization process, and I'm hoping for some guidance or suggestions on how to address it.

Abstract

The synchronization process frequently fails with a TypeError: Failed to fetch error. This failure seems to be related to the timing out of CouchDB's _changes requests when accessed through Cloudflare Tunnel, based on observations and logs.

Environment

Obsidian Version: 1.5.11
Plugin Version: 0.22.16
Server: self-hosted and accessed via Cloudflare Tunnel

Issue Detail

The process frequently fails with a TypeError: Failed to fetch error. While Developer Tools suggest a CORS policy issue blocking the _changes request to CouchDB, my observations indicate a potentially different cause.

Requests other than _changes to the same CouchDB instance do not face similar issues, and _changes requests consistently fail at exactly 100 seconds. Additionally, research into Cloudflare Tunnel's behavior points out an unofficial but widely noted timeout limit of 100 seconds.

These elements lead me to speculate that the disruptions are not due to CORS configuration but are likely caused by a timeout issue related to Cloudflare Tunnel.

Error Message from Developer Tools

Access to fetch at 'https://<couchdb_url>/_changes?style=all_docs&feed=longpoll&heartbeat=30000&filter=_selector&since=<since>&limit=25' from origin 'app://obsidian.md' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Cloudflare Tunnel Log Snippet

2024-03-25T06:37:24Z debug http POST https://<couchdb_url>/_changes?style=all_docs&feed=longpoll&heartbeat=30000&filter=_selector&since=<since>&limit=25 ...
2024-03-25T06:39:04Z error cloudflared Request failed {"connIndex":0,"dest":"https://<couchdb_url>/_changes?style=all_docs&feed=longpoll&heartbeat=30000&filter=_selector&since=<since>&limit=25","error":"context canceled","type":"http"}

Observations and Cloudflare Tunnel logs consistently show that each failure occurs exactly at the 100-second mark. This pattern suggests a timeout issue is at the heart of these frequent disruptions, rather than any misconfiguration with CORS.

Questions and Assistance Request

Given Cloudflare's 100-second timeout limit (which cannot be altered without an Enterprise plan), I'm seeking advice on potentially reducing the duration of the _changes requests made by the plugin. Are there configuration options or optimizations within the plugin or CouchDB settings that could help mitigate this frequent timeout issue?

I appreciate any guidance you can provide and am willing to assist with further debugging information if needed. Thank you for your time and effort in maintaining this valuable plugin.

Report materials

Report from the LiveSync

Report from hatch

---- Obsidian info ----
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.5.8 Chrome/120.0.6099.283 Electron/28.2.3 Safari/537.36
---- remote config ----
uuids:
  algorithm: sequential
  max_count: "1000"
cluster:
  n: "3"
  q: "2"
cors:
  credentials: "true"
  headers: accept, authorization, content-type, origin, referer
  max_age: "3600"
  methods: GET, PUT, POST, HEAD, DELETE
  origins: app://obsidian.md,capacitor://localhost,http://localhost
chttpd:
  backlog: "512"
  bind_address: any
  max_db_number_for_dbs_info_req: "100"
  max_http_request_size: "4294967296"
  port: "5984"
  prefer_minimal: Cache-Control, Content-Length, Content-Range, Content-Type, ETag, Server, Transfer-Encoding, Vary
  require_valid_user: "true"
  server_options: "[{recbuf, undefined}]"
  socket_options: "[{sndbuf, 262144}, {nodelay, true}]"
attachments:
  compressible_types: text/*, application/javascript, application/json, application/xml
  compression_level: "8"
admins: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
query_server_config:
  os_process_limit: "100"
  reduce_limit: "true"
vendor:
  name: The Apache Software Foundation
feature_flags:
  partitioned||*: "true"
chttpd_auth:
  authentication_redirect: /xxxxx/_utils/session.html
  require_valid_user: "true"
replicator:
  connection_timeout: "30000"
  http_connections: "20"
  interval: "60000"
  max_churn: "20"
  max_jobs: "500"
  retries_per_request: "5"
  socket_options: "[{keepalive, true}, {nodelay, false}]"
  ssl_certificate_max_depth: "3"
  startup_jitter: "5000"
  verify_ssl_certificates: "false"
  worker_batch_size: "500"
  worker_processes: "4"
ssl:
  port: "6984"
log:
  level: info
  writer: stderr
indexers:
  couch_mrview: "true"
couch_peruser:
  database_prefix: userdb-
  delete_dbs: "false"
  enable: "false"
httpd:
  WWW-Authenticate: Basic realm="couchdb"
  allow_jsonp: "false"
  authentication_handlers: "{couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}"
  bind_address: 127.0.0.1
  enable_cors: "true"
  enable_xframe_options: "false"
  max_http_request_size: "4294967296"
  port: "5986"
  secure_rewrites: "true"
  socket_options: "[{sndbuf, 262144}]"
ioq.bypass:
  compaction: "false"
  os_process: "true"
  read: "true"
  shard_sync: "false"
  view_update: "true"
  write: "true"
ioq:
  concurrency: "10"
  ratio: "0.01"
couch_httpd_auth:
  allow_persistent_cookies: "true"
  auth_cache_size: "50"
  authentication_db: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
  authentication_redirect: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
  iterations: "10"
  require_valid_user: "false"
  timeout: "600"
  secret: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
couchdb_engines:
  couch: couch_bt_engine
couchdb:
  attachment_stream_buffer_size: "4096"
  changes_doc_ids_optimization_threshold: "100"
  database_dir: ./data
  default_engine: couch
  default_security: admin_only
  file_compression: snappy
  max_dbs_open: "500"
  max_document_size: "50000000"
  os_process_timeout: "5000"
  single_node: "true"
  users_db_security_editable: "false"
  uuid: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
  view_index_dir: ./data

---- Plug-in config ---
version:0.22.16
couchDB_URI: self-hosted(HTTPS)
couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
liveSync: true
syncOnSave: false
syncOnStart: false
savingDelay: 200
lessInformationInLog: false
gcDelay: 0
versionUpFlash: ""
minimumChunkSize: 20
longLineThreshold: 250
showVerboseLog: false
suspendFileWatching: false
trashInsteadDelete: true
periodicReplication: false
periodicReplicationInterval: 60
syncOnFileOpen: false
encrypt: true
passphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
usePathObfuscation: true
doNotDeleteFolder: false
resolveConflictsByNewerFile: false
batchSave: false
deviceAndVaultName: ""
usePluginSettings: false
showOwnPlugins: false
showStatusOnEditor: true
showStatusOnStatusbar: true
showOnlyIconsOnEditor: false
usePluginSync: false
autoSweepPlugins: false
autoSweepPluginsPeriodic: false
notifyPluginOrSettingUpdated: false
checkIntegrityOnSave: false
batch_size: 25
batches_limit: 25
useHistory: true
disableRequestURI: true
skipOlderFilesOnSync: true
checkConflictOnlyOnOpen: false
showMergeDialogOnlyOnActive: false
syncInternalFiles: false
syncInternalFilesBeforeReplication: false
syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/
syncInternalFilesInterval: 60
additionalSuffixOfDatabaseName: edbb3b83325e880e
ignoreVersionCheck: false
lastReadUpdates: 22
deleteMetadataOfDeletedFiles: false
syncIgnoreRegEx: ""
syncOnlyRegEx: ""
customChunkSize: 50
readChunksOnline: true
watchInternalFileChanges: true
automaticallyDeleteMetadataOfDeletedFiles: 0
disableMarkdownAutoMerge: false
writeDocumentsIfConflicted: false
useDynamicIterationCount: false
syncAfterMerge: false
configPassphraseStore: ""
encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
permitEmptyPassphrase: false
useIndexedDBAdapter: true
useTimeouts: false
writeLogToTheFile: false
doNotPaceReplication: false
hashCacheMaxCount: 300
hashCacheMaxAmount: 50
concurrencyOfReadChunksOnline: 30
minimumIntervalOfReadChunksOnline: 25
hashAlg: xxhash64
suspendParseReplicationResult: false
doNotSuspendOnFetching: false
useIgnoreFiles: false
ignoreFiles: .gitignore
syncOnEditorSave: false
pluginSyncExtendedSetting: {}
syncMaxSizeInMB: 50
settingSyncFile: ""
writeCredentialsForSettingSync: false
notifyAllSettingSyncFile: false
isConfigured: true
settingVersion: 0

Obsidian debug info

Debug info

SYSTEM INFO:
	Obsidian version: v1.5.11
	Installer version: v1.5.8
	Operating system: Darwin Kernel Version 23.3.0: Thu Dec 21 02:29:41 PST 2023; root:xnu-10002.81.5~11/RELEASE_ARM64_T8122 23.3.0
	Login status: not logged in
	Insider build toggle: off
	Live preview: on
	Base theme: light
	Community theme: none
	Snippets enabled: 0
	Restricted mode: off
	Plugins installed: 3
	Plugins enabled: 2
		1: Code Editor Shortcuts v1.14.0
		2: Self-hosted LiveSync v0.22.16

RECOMMENDATIONS:
	Community plugins: for bugs, please first try updating all your plugins to latest. If still not fixed, please try to make the issue happen in the Sandbox Vault or disable community plugins.

Plug-in log

2024/3/25 16:40:13->LiveSync begin...
2024/3/25 16:40:17->Replication paused
2024/3/25 16:40:17->Replication paused
2024/3/25 16:41:58->HTTP:POST (36) to:/_changes?style=all_docs&feed=longpoll&heartbeat=30000&filter=_selector&since=98460-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYFh_5xcoBi7salhqoFpCrp6HCbksQBJhgYg9R9u0EEGsEGGZqYmRomJ6NqyAGsmLGM&limit=25 -> failed
2024/3/25 16:41:58->TypeError:Failed to fetch
2024/3/25 16:41:58->Replication paused
2024/3/25 16:42:00->Replication paused

Network log

homu-konamilk · Answer 1 · Mon Mar 25 2024 16:33:34 GMT+0800 (China Standard Time)

Sorry, I realized I misunderstood CouchDB's behavior for _changes requests. I see that when the feed=longpoll parameter is given, it waits until there is a change, instead of sending a response immediately.

I discovered a solution within the plugin's settings. By enabling the Use timeouts instead of heartbeats option, I was able to resolve the issue effectively.

My apologies for any confusion caused, and I hope this resolution might be of help to others who might face similar issues.