Templater Cursor Function (tp.file.cursor) stops working after 2^16 (65,536) characters
billy-bg opened this issue · comments
Plugin information:
- OS: macOS Ventura 13.4
- Templater version: 1.16.0
- Obsidian version: 1.3.5
- Templater settings: Template folder location: "Templates/", Timeout: "5", cursor jump key: "command+enter". (all settings found at bottom)
Templates used:
-<big><% tp.file.cursor(1) %></big>
above used to make text bigger. assigned hotkey is command+shift+>.
-[<% tp.file.cursor(1) %>](<% tp.file.cursor(2) %>)<% tp.file.cursor(3) %>
above used for link. assigned hotkey is command+J.
Describe the bug:
Whenever my document gets longer than 2^16 (65,536) characters, inserting a template does not allow the cursor functions (tp.file.cursor) to work as expected. In other words, my cursor does not start at tp.file.cursor(1) anymore (now starts at beginning of text). Using my cursor jump hotkey (command+enter), no longer causes the cursor to move at all (e.g. to tp.file.cursor(2), etc.).
Expected behavior:
Templater cursor functions work as expected if document they are inserted into is less than 2^16 (65,536) characters. Normally, inserting the second above template, the cursor should automatically be placed at tp.file.cursor(1). Pressing command+enter (cursor jump hotkey), causes the cursor to move to tp.file.cursor(2). Pressing it again (cursor jump hotkey), causes the cursor to move to tp.file.cursor(3).
Screenshots
Above screenshot shows a document with more than 2^16 (65,536) characters. Inserting the 1st template above places cursor incorrectly at the beginning of text.
Above screenshot shows a document with less than 2^16 (65,536) characters. Inserting the 1st template above places cursor correctly in the middle of the text.
Additional context
Inserting template with a hotkey or via templater drop down menu does not make a difference.
EDIT 16/7/23:
- Have tested this on a completely new vault with only the templater plugin installed and the issue still persisted. This confirms that this is most likely NOT due to interaction with other installed plugins.
- Tested in both source and live preview mode and issue still persists.
- Have tested other functions (such as tp.system.clipboard(), etc.) and these still work as expected. Issue seems to only exist with the tp.file.cursor() function specifically.
- Tested on multiple seperate documents, behaviour seems fairly consistent with document length before function no longer works.
- Tested an older release of Templater (1.3.0), the issue still persists with that version as well.
- Have found another user describing the exact same issue (still open after a year+): #674
- Have looked through code for the "Text Snippet" community plugin (works almost exactly the same way as the Templater Cursor Function). It appears that they have solved for the overload issue. I can easily see it being possible to appropriate that code into this plugin (changing a few things to match slight functionality differences). I am not good enough of a programmer to do it myself but maybe someone else here is.
- I am not sure if this helps, but in an attempt to fix the get_editor_position overflow issue (javascript string overflow once it exceeds value of 2^16), I have tried replacing the following code in main.js. The issue still persists however, and it seems that there may also be another issue elsewhere that i couldn't find:
get_editor_position_from_index(content, index) {
const substr = content.slice(0, index);
let l = 0;
let offset2 = -1;
let r = -1;
for (; (r = substr.indexOf("\n", r + 1)) !== -1; l++, offset2 = r)
;
offset2 += 1;
const ch = content.slice(offset2, index).length;
return { line: l, ch };
}
replace_and_get_cursor_positions(content) {
let cursor_matches = [];
let match;
const cursor_regex = new RegExp("<%\\s*tp.file.cursor\\((?<order>[0-9]{0,2})\\)\\s*%>", "g");
while ((match = cursor_regex.exec(content)) != null) {
cursor_matches.push(match);
}
if (cursor_matches.length === 0) {
return {};
}
cursor_matches.sort((m1, m2) => {
return Number(m1.groups && m1.groups["order"]) - Number(m2.groups && m2.groups["order"]);
});
const match_str = cursor_matches[0][0];
cursor_matches = cursor_matches.filter((m) => {
return m[0] === match_str;
});
const positions = [];
let index_offset = 0;
for (const match2 of cursor_matches) {
const index = match2.index - index_offset;
positions.push(this.get_editor_position_from_index(content, index));
content = content.replace(new RegExp(escape_RegExp(match2[0])), "");
index_offset += match2[0].length;
if (match2[1] === "") {
break;
}
}
return { new_content: content, positions };
}
WAS REPLACED WITH:
get_editor_position_from_index(content, index) {
const chunkSize = Math.pow(2, 16) - 1;
let chunkIndex = Math.floor(index / chunkSize);
let offsetInChunk = index % chunkSize;
let l = 0;
let ch = 0;
for (let i = 0; i <= chunkIndex; i++) {
let chunk = content.slice(i * chunkSize, (i + 1) * chunkSize);
if (i < chunkIndex) {
let newLinesInChunk = chunk.split("\n").length - 1;
l += newLinesInChunk;
} else {
let subChunk = chunk.slice(0, offsetInChunk);
let newLinesInSubChunk = subChunk.split("\n");
l += newLinesInSubChunk.length - 1;
ch = newLinesInSubChunk[newLinesInSubChunk.length - 1].length;
}
}
return { line: l, ch: ch };
}
replace_and_get_cursor_positions(content) {
let cursor_matches = [];
let match;
const cursor_regex = new RegExp("<%\\s*tp.file.cursor\\((?<order>[0-9]{0,2})\\)\\s*%>", "g");
while ((match = cursor_regex.exec(content)) != null) {
cursor_matches.push(match);
}
if (cursor_matches.length === 0) {
return {};
}
cursor_matches.sort((m1, m2) => {
return Number(m1.groups && m1.groups["order"]) - Number(m2.groups && m2.groups["order"]);
});
const match_str = cursor_matches[0][0];
cursor_matches = cursor_matches.filter((m) => {
return m[0] === match_str;
});
const positions = [];
let totalMatchLength = 0;
for (const match2 of cursor_matches) {
const index = match2.index - totalMatchLength;
positions.push(this.get_editor_position_from_index(content, index));
totalMatchLength += match2[0].length;
if (match2[1] === "") {
break;
}
}
let new_content = content;
for (const match2 of cursor_matches) {
new_content = new_content.replace(new RegExp(escape_RegExp(match2[0])), "");
}
return { new_content: new_content, positions };
}
Settings:
{
"command_timeout": 5,
"templates_folder": "Templates",
"templates_pairs": [
[
"",
""
]
],
"trigger_on_file_creation": true,
"auto_jump_to_cursor": true,
"enable_system_commands": true,
"shell_path": "",
"user_scripts_folder": "",
"enable_folder_templates": true,
"folder_templates": [
{
"folder": "",
"template": ""
}
],
"syntax_highlighting": true,
"enabled_templates_hotkeys": [
"Templates/big.md",
"Templates/code block.md",
"Templates/link.md",
"Templates/horizontal space break.md",
"Templates/inline code block.md",
"Templates/up arrow.md",
"Templates/down arrow.md",
"Templates/left arrow.md",
"Templates/right arrow.md",
"Templates/invisible character.md",
"Templates/embed.md"
],
"startup_templates": [
""
],
"enable_ribbon_icon": true
}