Giters

lucasrla / remarks

Extract annotations (highlights and scribbles) from PDF, EPUB, and notebooks marked with reMarkable tablets. Export to Markdown, PDF, PNG, SVG

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Notebooks and Quick Sheets support?

clement-elbaz opened this issue · comments

commented

Hello and thank you for your work on this project!

I'm trying to use my brand new remarkable V2 without the remarkable cloud and have my data stay within my local network at all time. So far it seems no open-source project provide a feature-complete solution to convert the entire xoshitl directory into a directory of well-located, well-named PDFs files. Your project seems to be the most mature solution to get there, as you already handle the annotated PDF use-case, arguably the most complex. Handling notebooks and quick sheets seem doable in comparison.

Would you consider that "in scope" of your project and accept pull requests in that regard? (assuming the pull requests themselves are decent enough of course) Or should I consider contributing to other projects for this use case? Thanks !

commented

Hey @clement-elbaz, thanks for reaching out.

Just to make sure I fully understood what you mean by Notebooks and Quick Sheets support.

  1. Is it the ability to treat Notebooks/Quick Sheets as "inputs" and export them to PDF and/or PNG? (If yes, what about EPUB as well?)

  2. Is it the ability to "rsync" the contents inside the xochitl directory to a computer?

  3. All of the above?

  4. Something else?

I am open to either (1) and/or (2), and happy to review and comment on your PRs.

Re (1), it should be very doable to extend the current codebase to support Notebooks/Quick Sheets. And it is a very natural extension. (As for EPUB, I am not sure, I haven't looked into its details yet.)

If it is (2), what do you have in mind? A wrapper around scp? Else?

Thanks

commented

Thank you for the fast answer!

Personally I would be interested in tackling (1) only, without EPUB support at first. This feels like a goal I can realistically aspire to complete in a reasonable amount of time.

As a preliminary analysis, the main architectural change introduced by (1) would be the need to access a copy of the template directory ( /usr/share/remarkable/templates ). Indeed, in order to get a complete picture of the document as it is showed on the screen, one need to superimpose the SVG created by rm2svg with the corresponding SVG template file. It seems unpractical to keep a copy of the templates directory in the codebase, both for copyright reasons and because they might change at every OS update. Therefore, the best plan I can come up with is to introduce an optional parameter for the user to provide the path of a copy of the templates directory. What do you think?

This new requirement to access or copy both the xochitl and templates directories from the remarkable create additional complexity for the user that might justify doing something about (2) in the long term. However to me this feels secondary to (1) for the time being.

If you're OK with this plan I will explore implementing (1) (without EPUB) in the next few days/weeks and get back to you here if I encounter any hurdle. Thank you again!

commented

Great! Sounds like a plan.

The solution you are proposing for accessing the templates seems very reasonable to me.

Looking forward to your contributions. Thanks!

commented

One suggestion has just occurred to me. If no --templates_dir is provided by the user, we could guess where they have their templates' .png and .svg before throwing an error.

This would also make the feature nearly perfectly backwards compatible with the existing code and usage patterns.

For example:

# let's suppose the user enters just
python -m remarks ~/backups/xochitl/ output/

# ideally we would first look for templates image files both
# - in ~/backups/templates (because it is the "natural candidate", it being a sibbling of ~/backups/xochitl/)
# - in the current directory (a.k.a. `pwd`)

# if the user sets the template directory explicitly, we would go straight to it, of course
python -m remarks ~/backups/xochitl/ output/ --templates_dir ~/metadata/templates

Thanks again!

commented

This is a good idea, and should not add much additional complexity.

I was wondering a bit about potential "false positives" (detecting and trying to use an existing but incorrect directory that does not contain the proper SVG files) but an incorrect directory is not likely to have files sharing the names of the SVG template files. Therefore as long as the existence check for individual template files is robust, misdetecting the directory should not be a big problem in practice. I'll try to support that!

commented

I hacked in support for this a while ago, just pushed it to https://github.com/Gigahawk/remarks/tree/gigahawk_dev.

It needs a lot of work to be worth merging in, but it was mostly to get something working right away.
I never bothered to deal with templates, personally I prefer to only have the template present when I'm actually writing the document on the tablet, not in the export.

I'll probably start chipping away at reimplementing it in a less awful way in the coming days, but as a POC feel free to take a look at it.

commented

Nice, @Gigahawk! I hope to have the time to look at it by next week.

BTW, having templates as an optional background does sound like a good idea.

commented

@Gigahawk: cool! Two weeks ago I also started a new code change for handling notebooks (without template support for the time being). But I've had trouble finding time to finish it.

If you're faster than me I'll happily trash my current implementation and add templates support on top of yours. In the meantime I'll continue the work on mine and submit it when it's ready.

commented

Hi, what is necessary to merge this?
I may be able to help.

commented

Hey @Phill93, thanks for reaching out!

I am happy to merge any PR that works (and whose code looks reasonably clear).

I believe @Gigahawk started implementing it back in January Gigahawk@5454356 But their fork has been stale for a while.

Thanks

commented

This would be amazing!

I generally prefer taking notes on the reMarkable tablet while in meetings.
If text notes can be formatted in Markdown to share with colleagues (GitHub, etc.) would be great. 💪

The trick is also supporting the variety of the markdown formatting options (tables, indentations, etc.) and being fairly handwriting recognition, so you don't have to fix more than 1-2 errors per page.

e.g. of a table
https://www.reddit.com/r/RemarkableTablet/comments/necro0/would_be_great_if_the_text_conversion_could_also/

Ideally, there would a GUI tool on your computer (I can give this a try) where you see the LiveView and the markdown being converted live :)
In that way, you can adjust your writing on the reMarkable, rather than fixing the issues after. But this could for version 2.0 😄

commented

Hey there, I needed Notebooks and Quick Sheets support, so I modified remarks to handle these. I made many changes at once, but the idea is simple (if no PDF path is available, open a blank A4 document instead). There is also a change to handle pages inserted into PDFs.

You might want to look through my fork (edit: deleted) and pull out these updates!

commented

Nice, @apoorvkh! Will have a look. If you'd like to submit anything as a PR, happy to review and merge it as well. Thanks

commented

Hey all, I've just pushed a commit that adds support to Notebooks and in-document Note Pages.

Thanks for showing me the way, @apoorvkh!

commented

Happy for this feature to have been completed, and sorry for not finding the time to carry it through myself. Thank you @apoorvkh and @lucasrla !

commented

This is great, thanks lucasrla! Glad you were able to take a look and get it implemented. (I'm traveling, so a PR would have taken me much longer to get to.)