whacked / CalibrePluginScaleATon

A skeleton that "tries" to be a basis for practical calibre plugin creation

CalibrePluginScaleATon

CalibrePluginScaleATon

A skeleton that “tries” to be a basis for practical calibre plugin creation.

memo

github’s org-mode renderer will not render this correctly!

where we are

lsb_release -a

Distributor ID:	Ubuntu
Description:	Ubuntu 12.04.1 LTS
Release:	12.04
Codename:	precise

calibredb --version

calibredb (calibre 0.9.11)

python -c 'import sys; print sys.version_info'

sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0)

where we want to go

• add a menu entry to right click that says “Get info”
• when you click it, it prints the dict of the book entry

version 1

ref: http://manual.calibre-ebook.com/creating_plugins.html

first code in the tutorial involves importing calibre:

import os
from calibre.customize import FileTypePlugin

Traceback (most recent call last):
  File "<stdin>", line 7, in <module>
  File "<stdin>", line 4, in main
ImportError: No module named calibre.customize

So we need to set up calibre as an importable module…

set up our development environment

ref: http://manual.calibre-ebook.com/develop.html

set up a virtualenv

virtualenv venv

New python executable in venv/bin/python
Installing setuptools............done.
Installing pip...............done.

get calibre src for python import

make sure we have a copy of the bazaar code somewhere. skipping this part – you want to follow develop.html linked above.

mine is sitting in ~/dev/calibre/calibre-src; make sure you run bzr merge to update it.

grep -n numeric_version ~/dev/calibre/calibre-src/src/calibre/constants.py

7:numeric_version = (0, 9, 11)
8:__version__   = u'.'.join(map(unicode, numeric_version))

.pth file hacks for calibre import

cat venv/lib/python2.7/site-packages/calibre.pth

import os, sys; sys.path.append(os.path.expanduser("~/dev/calibre/calibre-src/src"))
import os, sys; sys.resources_location = os.path.expanduser("~/dev/calibre/calibre-src/resources")
import os, sys; sys.extensions_location = os.path.expanduser("~/dev/calibre/calibre-src/src/calibre/plugins")
import sys; sys.executables_location = "/usr/bin"

now attempt import calibre

. venv/bin/activate
python -c 'import calibre'

Loading ICU failed with:  No module named icu
Loading ICU failed with:  No module named icu

install PyICU

. venv/bin/activate
pip install PyICU

Downloading/unpacking PyICU
  Running setup.py egg_info for package PyICU

Installing collected packages: PyICU
  Running setup.py install for PyICU
    building '_icu' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c numberformat.cpp -o build/temp.linux-x86_64-2.7/numberformat.o -DPYICU_VER="1.5"
    # ... #
    # ... # elided
    # ... #
    g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-x86_64-2.7/numberformat.o build/temp.linux-x86_64-2.7/format.o build/temp.linux-x86_64-2.7/unicodeset.o build/temp.linux-x86_64-2.7/bases.o build/temp.linux-x86_64-2.7/normalizer.o build/temp.linux-x86_64-2.7/tzinfo.o build/temp.linux-x86_64-2.7/layoutengine.o build/temp.linux-x86_64-2.7/charset.o build/temp.linux-x86_64-2.7/locale.o build/temp.linux-x86_64-2.7/iterators.o build/temp.linux-x86_64-2.7/collator.o build/temp.linux-x86_64-2.7/common.o build/temp.linux-x86_64-2.7/calendar.o build/temp.linux-x86_64-2.7/dateformat.o build/temp.linux-x86_64-2.7/search.o build/temp.linux-x86_64-2.7/_icu.o build/temp.linux-x86_64-2.7/regex.o build/temp.linux-x86_64-2.7/transliterator.o build/temp.linux-x86_64-2.7/errors.o -licui18n -licuuc -licudata -licule -o build/lib.linux-x86_64-2.7/_icu.so

Successfully installed PyICU
Cleaning up...

. venv/bin/activate
python -c 'import calibre; print "OK"'

OK

there was actually a problem before, where src/calibre/utils/icu.py printed “icu not ok”. I placed a print _icu after the if _icu is None test, and the error went away. presumably something was stale and updating it reloaded something else that propagated the fix.

test out our environment

. venv/bin/activate
python -c 'import os; from calibre.customize import FileTypePlugin; print "OK"'

OK

test the HelloWorld plugin

mkdir HelloWorldPlugin

(run C-v-t or (org-babel-tangle) to generate this file)

import os
from calibre.customize import FileTypePlugin

class HelloWorld(FileTypePlugin):

    name                = 'Hello World Plugin' # Name of the plugin
    description         = 'Set the publisher to Hello World for all new conversions'
    supported_platforms = ['windows', 'osx', 'linux'] # Platforms this plugin will run on
    author              = 'Acme Inc.' # The author of this plugin
    version             = (1, 0, 0)   # The version number of this plugin
    file_types          = set(['epub', 'mobi']) # The file types that this plugin will be applied to
    on_postprocess      = True # Run this plugin after conversion is complete
    minimum_calibre_version = (0, 7, 53)

    def run(self, path_to_ebook):
        from calibre.ebooks.metadata.meta import get_metadata, set_metadata
        file = open(path_to_ebook, 'r+b')
        ext  = os.path.splitext(path_to_ebook)[-1][1:].lower()
        mi = get_metadata(file, ext)
        mi.publisher = 'Hello World'
        set_metadata(file, mi, ext)
        return path_to_ebook

this file should be runnable from the venv command line (producing no output)

install the HelloWorld plugin

calibre-customize -b HelloWorldPlugin

Plugin updated: Hello World Plugin (1, 0, 0)

what that did:

how to poke around in the REPL

we want to play with self.gui within the InterfaceAction class method

looking around, there are a couple ways of achieving this:

ingress

http://pypi.python.org/pypi/ingress/0.1.1 didn’t try this

ipython

http://stackoverflow.com/questions/11513132/embedding-ipython-qt-console-in-a-pyqt-application tried this. unable to find good workaround for ValueError: API 'QString' has already been set to version 1

twisted.manhole

this worked, covered below

install twisted

for the sake of completeness we’ll install twisted via pip:

. venv/bin/activate
pip install twisted

Downloading/unpacking twisted
  Downloading Twisted-12.3.0.tar.bz2 (2.6MB): 2.6MB downloaded
  Running setup.py egg_info for package twisted
    
Downloading/unpacking zope.interface>=3.6.0 (from twisted)
  Downloading zope.interface-4.0.2.tar.gz (139kB): 139kB downloaded
  Running setup.py egg_info for package zope.interface
    
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./venv/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg (from zope.interface>=3.6.0->twisted)
Installing collected packages: twisted, zope.interface
  Running setup.py install for twisted
    
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o

    # ... #
    # ... # elided
    # ... #

Successfully installed twisted zope.interface
Cleaning up...

running calibre-debug

when you start calibre-debug you need to set the CALIBRE_DEVELOP_FROM variable.

without modifying the virtualenv init script, this ad-hoc call works:

. venv/bin/activate
CALIBRE_DEVELOP_FROM=../calibre-src/src calibre-debug -g

but if we put e.g. import twisted at the top of calibre-src/src/calibre/__init__.py and run that, we get:

. venv/bin/activate
CALIBRE_DEVELOP_FROM=../calibre-src/src calibre-debug -g

...
ImportError: No module named twisted

python __init__.py hack for zope.interface

we get ImportError: Twisted requires zope.interface 3.6.0 or later: no module named zope.interface.

from hacking around the paths and forcing an unexpected import; to workaround that:

touch venv/lib/python2.7/site-packages/zope/__init__.py

path hacks to make calibre load our venv packages

hacked the sys.path directly:

head ../calibre-src/src/calibre/__init__.py

import os, sys
print "this is the modified __init__.py in calibre-src"
print sys.version


sys.path.extend([
    os.path.abspath('venv/lib/python2.7/site-packages'),
                 ])

import twisted

then it starts, and you can import twisted from within the plugin.

then modify the plugin to launch a telnet manhole

using the manhole

Start calibre-debug and click our plugin menu from the context menu. After you dismiss the popup window, calibre freezes as the telnet server launches.

At this point, you can telnet into our server, hit ENTER past the username/password, and poke around:

 ➭ rlwrap telnet localhost 2222
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

twisted.manhole.telnet.ShellFactory
Twisted 11.1.0
username: 
password: *****
>>> gui
<calibre.gui2.ui.Main object at 0x7f16b41cb710>
>>> reactor.stop()
Connection closed by foreign host.
 ➭ 

calling reactor.stop() cedes control back to calibre. You can also Ctrl-C from the terminal where you launched calibre.

install see to help us poke around

. venv/bin/activate
pip install termcolor see

Downloading/unpacking termcolor
  Downloading termcolor-1.1.0.tar.gz
  Running setup.py egg_info for package termcolor

Downloading/unpacking see
  Downloading see-1.0.1.tar.bz2
  Running setup.py egg_info for package see

Installing collected packages: termcolor, see
  Running setup.py install for termcolor

  Running setup.py install for see

Successfully installed termcolor see
Cleaning up...

connecting from emacs

modify the prompt pattern:

(setq telnet-prompt-pattern "^[^#$%>\n]*>>> ")

Use C-u M-x telnet to connect with prompt for port number

Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

twisted.manhole.telnet.ShellFactory
Twisted 12.3.0
username: 
password: *****
>>> from see import see
>>> see(self.gui.library_view)
    .*                        hash()                    repr()
    str()                     .AboveItem                .AllEditTriggers
    .AnimatingState           .AnyKeyPressed            .BelowItem
    .Box                      .CollapsingState          .ContiguousSelection
    .CurrentChanged           .CursorAction()           .DoubleClicked
    .DragDrop                 .DragDropMode()           .DragOnly
# ... #
# ... # elided
# ... #
>>> self.gui.library_view.get_selected_ids()
[1863]
>>> self.gui.library_view.current_id
1863
>>> m = self.gui.library_view.model()
>>> m
<calibre.gui2.library.models.BooksModel object at 0x3a81b00>
>>> see(m)
    .*                         hash()                     repr()
    str()                      .about_to_be_sorted()      .add_books()
    .add_catalog()             .add_news()                .alignment_map
    .authors()                 .beginInsertColumns()      .beginInsertRows()
    .beginMoveColumns()        .beginMoveRows()
# ... #
# ... # elided
# ... #
>>> help(m.authors)
Help on method authors in module calibre.gui2.library.models:

authors(self, row_number) method of calibre.gui2.library.models.BooksModel instance

>>> m.authors(10)
u'calibre'
>>> data = m.get_book_info(10)
>>> data.author
[u'Unknown']

we need to convert between some calibre id and the Qt table row number

>>> help(self.gui.library_view.ids_to_rows)
Help on method ids_to_rows in module calibre.gui2.library.views:

ids_to_rows(self, ids) method of calibre.gui2.library.views.BooksView instance

>>> self.gui.library_view.ids_to_rows([10])
OrderedDict([(10, 1243)])
>>> self.gui.library_view.ids_to_rows([10,11,99999,1863])
OrderedDict([(1863, 0), (10, 1243), (11, 1244)])
>>> 
>>> help(m.get_book_info)
Help on method get_book_info in module calibre.gui2.library.models:

get_book_info(self, index) method of calibre.gui2.library.models.BooksModel instance
>>> info = m.get_book_info(0)
>>> info.id
1863
>>> print m.get_book_info(21)
Title               : The Economist [Fri, 16 Nov 2012]
Title sort          : Economist [Fri, 16 Nov 2012], The
Author(s)           : calibre [calibre]
Tags                : News, The Economist
Timestamp           : 2012-11-16T19:30:06+00:00
Published           : 2012-11-16T19:30:06+00:00
>>> info.all_field_keys()
frozenset(['rating', u'#issue', 'title_sort', 'application_id', 'pubdate', # ...
>>> info = m.get_book_info(21)
>>> info.get('title_sort')
u'Economist [Fri, 16 Nov 2012], The'
>>> 

and we have what we need to know.

>>> reactor.stop()
Connection closed by foreign host.

Process telnet-localhost:2222 exited abnormally with code 1

Right-click menu plugin

mkdir MyPlugin

plugin-import-name-myplugin.txt

calibre likes this text file to be empty, but I like to put some install memo in it

calibre-customize -b MyPlugin

Then you can call sh MyPlugin/*.txt to deploy it locally. For now we’ll do this. Later, we might change it to run the zip -r command for bundling

__init__.py

from calibre.customize import InterfaceActionBase

class MyPlugin(InterfaceActionBase):

    name                = 'Right click plugin'
    description         = 'Create an action menu that appears on right click'
    supported_platforms = ['windows', 'osx', 'linux']
    author              = 'Sir Skeleton'
    version             = (0, 0, 1)
    minimum_calibre_version = (0, 7, 53)

    actual_plugin       = 'calibre_plugins.myplugin.ui:RightClickPlugin'

    def is_customizable(self):
        return True

    def config_widget(self):
        from calibre_plugins.myplugin.config import ConfigWidget
        return ConfigWidget()

    def save_settings(self, config_widget):
        '''
        Save the settings specified by the user with config_widget.

        :param config_widget: The widget returned by :meth:`config_widget`.
        '''
        config_widget.save_settings()

        # Apply the changes
        ac = self.actual_plugin_
        if ac is not None:
            ac.apply_settings()
        

ui.py

ref: http://blog.vrplumber.com/index.php?/archives/1631-Minimal-example-of-using-twisted.manhole-Since-it-took-me-so-long-to-get-it-working….html

from calibre.gui2.actions import InterfaceAction
from calibre.gui2 import question_dialog, info_dialog

class RightClickPlugin(InterfaceAction):

    name = 'Right Click Menu'

    action_spec = ('Right Click Menu', None,
                   'Activate the menu', None) # None = no keyboard shortcut

    action_type = 'current'

    def genesis(self):
        # skip the icon creation
        # icon = get_icons('images/icon.png')
        # self.qaction.setIcon(icon)
        self.qaction.triggered.connect(self.show_dialog)

    def show_dialog(self):
        # The base plugin object defined in __init__.py
        base_plugin_object = self.interface_action_base_plugin
        # Show the config dialog
        # The config dialog can also be shown from within
        # Preferences->Plugins, which is why the do_user_config
        # method is defined on the base plugin class
        do_user_config = base_plugin_object.do_user_config

        # self.gui is the main calibre GUI. It acts as the gateway to access
        # all the elements of the calibre user interface, it should also be the
        # parent of the dialog
        m = self.gui.library_view.model()
        selected_ids = self.gui.library_view.get_selected_ids()
        id_rows      = self.gui.library_view.ids_to_rows(selected_ids)
        if len(selected_ids) is 0: return
        
        retrieve_id = id_rows[selected_ids[0]]
        info = m.get_book_info(retrieve_id)
        str_info = "\n".join(map(lambda k: "%s: %s" % (k, info.get(k)), ['application_id', 'title', 'authors', 'timestamp']))
        info_dialog(self.gui, "Item info", str_info, show=True)
        
        # here's the telnet manhole
        from twisted.internet import reactor
        from twisted.manhole import telnet
        
        context = locals()
        
        def createShellServer():
            factory = telnet.ShellFactory()
            port = reactor.listenTCP(2222, factory)
            factory.namespace = context
            factory.username = ''
            factory.password = ''
            return port
       
        if question_dialog(self.gui, "telnet manhole section", "start shell server?"):
            reactor.callWhenRunning(createShellServer)
            reactor.run()

    def apply_settings(self):
        from calibre_plugins.myplugin.config import prefs
        # In an actual non trivial plugin, you would probably need to
        # do something based on the settings in prefs
        prefs

config.py

from PyQt4.Qt import QWidget, QHBoxLayout, QLabel, QLineEdit

from calibre.utils.config import JSONConfig

# You should always prefix your config file name with plugins/,
# so as to ensure you dont accidentally clobber a calibre config file
prefs = JSONConfig('plugins/myplugin')

# Set defaults
prefs.defaults['my_msg_header'] = 'Your book info:'

class ConfigWidget(QWidget):

    def __init__(self):
        QWidget.__init__(self)
        self.l = QHBoxLayout()
        self.setLayout(self.l)

        self.label = QLabel('Message header:')
        self.l.addWidget(self.label)

        self.msg = QLineEdit(self)
        self.msg.setText(prefs['my_msg_header'])
        self.l.addWidget(self.msg)
        self.label.setBuddy(self.msg)

    def save_settings(self):
        prefs['my_msg_header'] = unicode(self.msg.text())

Actually getting the right click menu to show up

turns out, there isn’t an API to create a context menu. You add it via:

1. Preferences
   • 
2. Toolbar
   • 
3. … books in the calibre library
   • 
4. and move it to the right pane.
   • 

testing the right click menu

now a right click gives this menu:

with this popup:

after you click OK: