Feature Request: Option to emit relative paths for "file" field in Bib(La)Tex export
mhminai opened this issue · comments
In Short
When we export a bibtex without exporting the files, the path in the "file" field is absolute. Which does make a lot of sense as we can possible save this exported bibtex anywhere.
Would it be possible to emit the relative paths for (at least) those attachments which are located in the "Base Directory" option of Zotero. Internally Zotero does appear to use relative paths for links to files that are in the tree of this directory.
It seems that this is possible as some discussion in Issue #21 shows.
In Detail
I am using Zotero as a reference manager while finetuning a workflow for writing academic papers using pandoc. Pandoc requires a bibtex file for reference management, hence the need to export from Zotero to bibtex. I also intend to use docear, possibly from a different computer.
I am using Zotero in combination with zotfile, zotero-better-bibtex and zotero-bib-autoexport. Zotero allows me to quickly retrieve metadata for my PDFs. Then zotfile allows me to move all the PDFs to a common folder (say "Literature") and renaming them to "AuthorYYYY.pdf", while modifying the zotero entry to have a link to the file. I then use BBT to generate bibtex keys compatible with the filename (an option for this could also be a cool feature, but I digress). Now if the bibtex had relative paths, I could simply pop the generated .bib file in my Literature folder, and Voila, I would have a self contained folder with a .bib file having the PDFs linked, while still retaining all functionality of Zotero itself.
This allows me to put the Literature folder on dropbox and use (view, no point in modifying the bibtex as it would get overwritten by zotero-bib-autoexport) it from any computer with basic bibtex support. Also allows docear to function seamlessly with Zotero.
Considerations
- I would like to avoid multiple copies of the PDF at all costs, if possible. I use my home and lab computers almost equally.
- A WebDAV account is not an option I would like to consider at the moment.
- I hate zotero's policy of having PDF files nested really deep in some structure. I would like to keep my literature as I please.
Thanks a lot for considering this feature request. I am willing to help with any coding/testing if required, though I know awful little about coding.
No problems. There is already a feature in the current release (which does not yet have a GUI, I just now notice) that does the basics of this; if you go into about:config and enable the "attachmentRelativePath" option, the paths will just be "files/[filename]". If I understand you correctly, this does what you want, but you want "files" to be the basedir. Correct?
I would like there to be no "files" at all.
There is a concern with using [filename] by itself. Reason being when exported with files, which is when I expect this option would be enabled, Zotero creates a new directory tree under a folder "files", which I guess is done by Zotero and simply used by the plugin, so in that case [filename] would still contain that directory structure under files/
If that is the case, It would break everything. The structure under basedir would not be preserved and thus all files would end up being missing.
I think the proper solution would be as follows:
- Have an option for BBT which says "Use relative paths for files under basedir" which is a checkbox
- If this checkbox is disabled do whatever BBT is doing right now
- If this checkbox is enabled, check if absolute filename starts with basedir
- if it does, truncate basedir from absolute filename and emit resultant (which might still have some directory structure)
- if it doesn't emit filename as is.
This would make it consistent with what I think the internal representation for linked files is in Zotero, right now.
No, translator decides where the files go (even if Zotero does suggest the path to use). The option I mentioned makes that path "files/[whatever the filename happens to be]", and I now remember why I didn't make a GUI for it - with the simplified filename comes the risk of duplicate filenames being generated. No directory structure at all expect "files" when that option is enabled.
That option you outline is reasonable though, so I'll look into that.
I checked with enabling this option. The file name I get in the bibtex is:
file = {Zhang2010.pdf:files/161/Zhang2010.pdf:application/pdf}
The "161" is what I think zotero suggests to the plugin and indeed when I export with the "Export Files" option in "Translator Options" ticked I get a folder ("Exported Folder") with the following structure:
- "Exported Folder"
- "bibfilename".bib
- files
- 161
- Zhang2010.pdf
- 161
In this case the bib file is pointing to the correct Zhang2010, but I now have a duplicate PDF, which was exported with the bibfile. Now if I open this with jabref and click on the PDF link I go to the new PDF, causing all sorts of problems if I unfortunately annotate it there.
My folder is as follows:
- "literature_folder"
- Zhang2010.pdf
- "other_pdfs"
- "possibly a sub folder"
- "Another2010.pdf"
I would like to export the bibtex file and save it in the top level as so:
- "literature_folder"
- "filename".bib
- Zhang2010.pdf
- "other_pdfs"
- "possibly a sub folder"
- "Another2010.pdf"
The path till "literature_folder" is given to Zotero as "basedir", so if there was an option to emit filenames in the bibtex file relative to "basedir", it would all be hunky dory. I am okay with not permitting sub folders, so that the new option simply puts the filename and the user has to ensure that he keeps the bib file in a flat directory with all the PDFs
Thanks for the fantastically fast responses. Do you want me to look at some code or something to make myself clearer?
Ah no, the ID in the path is put there by me to make the filenames unique. The translator does have full control over the path.
I think in fact if you have basedir selected and you opt not to export files (only names), just defaulting to basedir-relative names would be the sensible way to go, no extra options in the prefs. This is what you need, right? And only in this circumstance?
I think I have a better idea; if the path of the attachment is under the target directory of the exported file, I'll just always opt for the relative path. Should be fail-save (after all, the paths can't clash), and relatively easy to implement.
That should be great. As long as we are able to determine the target directory, before hand.
The target directory is simply the directory where the BibTeX file is exported to.
Hello,
Just saw this today. And tried it on a Mac (OSX Yosemite). The export still seems to have absolute paths. My Literature folder is arranged by Journal/Year, and I am exporting to the top level. As per your comment of 22nd December, I was expecting the paths to be relative from the top level folder.
Example:
Literature
|
|----------- Journal of Management
| | -------------------- 2005
. | ---------------- ABC.pdf
I was expecting to see the file path: ./Journal of Management/2005/ABC.pdf
What I see is the full path from /Users/XYZ/Download/Literature/Journal of Management/2005/ABC.pdf
Do I need to make any changes to the preferences?
Sorry, yes, this has an as of yet GUI-less preference called extensions.zotero.translators.better-bibtex.attachmentRelativePath, which needs to be set to true for this to work.
Hello,
The feature doesn't seem to work as of now. Below I give what I did and what happened.
I disabled the version of BBT I had, and installed the one you have linked to (0.7.27), with this I get an error "An error occurred while trying to export the selected file", this happens whether the GUI-less preference is TRUE or FALSE. i.e. it makes no difference. I have pasted the error I get at the end of this message.
With the latest version installed (0.7.33), I get the absolute path with the preference set to FALSE, as expected, and with the preference set to TRUE, I get the path as if I had exported with the "Export Files" checkbox selected. I get:
file = {Yu.pdf:files/666/Yu.pdf:application/pdf}
I have shortened the name of the file so as to make it clear.
Sorry it took me so long to test, but I am stuck with an analysis. Do let me know what else I can check.
The lines between the dashes are what the "zotero error report" has. I get this by selecting Help -> Report Errors to Zotero.
[JavaScript Error: "[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.createStatement]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: Zotero.DBConnection.prototype.getStatement :: line 298" data: no] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID]"]
[JavaScript Error: "[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.createStatement]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: Zotero.DBConnection.prototype.getStatement :: line 298" data: no] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID]"]
[JavaScript Error: "[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.createStatement]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: Zotero.DBConnection.prototype.getStatement :: line 298" data: no] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID]"]
[JavaScript Error: "[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.createStatement]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: Zotero.DBConnection.prototype.getStatement :: line 298" data: no] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID] [QUERY: select citekey, citeKeyFormat from keys where itemID=? and libraryID = ?] [ERROR: no such column: libraryID]"]
version => 4.0.26.1, platform => Win32, oscpu => Windows NT 6.3; WOW64, locale => en-US, appName => Zotero, appVersion => 4.0.26.1, extensions => Zotero automatic export (1.1.9.1, extension), ZotFile (4.1.3, extension), Zotero LibreOffice Integration (3.5.9.SA.4.0.26.1, extension), Zotero Word for Windows Integration (3.1.18.SA.4.0.26.1, extension), Zotero Better Bib(La)Tex (0.7.27, extension), Google Update (1.3.26.9, plugin), Shockwave Flash (16.0.0.305, plugin), Java(TM) Platform SE 8 U31 (11.31.2.13, plugin), Java Deployment Toolkit 8.0.310.13 (11.31.2.13, plugin), Adobe Acrobat (11.0.10.32, plugin), PDF-XChange Viewer (2.5.311.0, plugin), VLC Web Plugin (2.1.3.0, plugin), Microsoft Office 2013 (15.0.4420.1017, plugin), Microsoft Office 2013 (15.0.4420.1017, plugin)
Can you check if the updated release does what you want?
Even though I am subscribed to notifications, I am not getting any email when you update this issue, I need to check back to find that you had updated this 8 days ago. :-)
I will check and update this issue today itself.
With version 0.8.40 of BBT. It still doesn't seem to work.
When I set extensions.zotero.translators.better-bibtex.attachmentRelativePath to false, I get absolute paths as expected, when I set it to true, I still get absolute paths. There appears to be no difference in the bibtex file generated.
I also tried exporting just a single item as bibtex to the same directory that contained the PDF file. Still I got the absolute path.
I can't think of how I could help you debug this issue. Thanks a lot for working on it.
In case the option of "relative directory when path of document is under path of target directory of exported file" appears not easy, you could possibly try the "relative to Base Directory option" given in zotero itself. This is under Preferrences -> Advanced -> Files and Folders -> Linked Attachment Base Directory
OK, I think I have replicated your setup, and I think it works now. I've just pushed it out as .41
Sir. You have just blown me away. :-) Works exactly as expected. When the option is set to true, the bib file contains relative paths as long as the document is under the directory to which I export, otherwise it gives me absolute paths. With the option set to false it gives the absolute path. Great work!
Thanks a lot. This issue can now be considered closed. You might want to document the option and the behavior though, if you plan on making it generally available. Let me know if you need help in the documentation.
p.s. I have currently tried with a collection of a few items and an individual item.
I believe I can close the issue now?
I'm actually considering making this the fixed behavior rather than adding another option. It is only triggered when you don't save the attachments and they happen to live under the directory the export is under; I can't imaging why you'd want to change this. When I decide on this, the hidden preference will just go away.
I am not getting this to work (I am using BBT v. 1.0.19 with standalone Zotero). Option attachmentRelativePath does not exist, which I think is expected based on the thread above. However, I am having the files with the complete path.
The bib file is being exported to /home/ramon/Zotero-data/storage/zotero.bib. And that is where Zotero places all the attachements. For example, one entry in the bibfile shows
file = {Attachment:/home/ramon/Zotero-data/storage/J5TSKG2N/Schmidt2010-gpzzggah.pdf:application/pdf}
I was expecting that the /home/ramon/Zotero-data/storage would not be there. (In case it matters, but I think it shouldn't, storage is a symbolic link to another location, which is synced between computers, so I keep the database in sync via Zotero, but the attachements themselves, and the bib file, by other means ---syncthing, actually).
Looking at some of the other options under about:config I have extensions.zotero.saveRelativeAttachmentPath but setting it to true or false makes no difference.
What am I doing wrong?
The behaviour has since become the default behaviour, so the option is no longer there. Sorry for the late reply -- I don't commonly review closed issues. If you are still experiencing this on 1.2.34, could you please open a new issue?
I've got all of this working (including the renaming of filenames to match the citekey) so that the desired workflow at the top ALMOST works perfectly.
If my export directory is ExpDir, my bibfile is Exp.bib, and the citekey-and-pdf-name is myPaper, the entry for myPaper in Exp.bib is
File = {MyPaper.pdf:/MyPaper.pdf:application/pdf}
which JabRef interprets literally as saying that the file is in my root directory /.
This can be fixed by hand by adding the comment
@comment{jabref-meta: fileDirectory:.;}
to the Exp.bib file. But I can't figure out any way to automatically add that comment to the file. And, having to do it by hand every time I (automatically) export, or every time I use the bib file, kind of defeats the purpose (which is to have a fully automatic process).
I guess this seems like maybe a question for the ZotFile developer, but the context comes from this thread so I'm posting here.
No, that's an error in BBT I think. Zotfile doesn't affect the paths exported by BBT. I'm a little surprised by the generated path though; are you exporting the bib file to a location within the zotero storage directory? If so, I'd still expect the paths to be <giberish>/myPaper.pdf.
Could you open a new issue for this please?
For what is worth, and using v. 1.4.2, I am getting the full path (which is what I actually really want). I am exporting to /home/ramon/Zotero-storage/zotero.bib. Under ~/Zotero-data I have a directory storage which is a symlink to /home/ramon/Zotero-storage/. So I get paths as /home/ramon/Zotero-data/storage/KUC65AEA/somefile.pdf. The point is that the export, in BBT, is not a file explicitly under the Zotero-data directory, but to a different one (Zotero-storage), even if this is symlinked from storage under zotero-data.
I don't think there's any way I can prevent that. I don't get told about symlinks.
@llorracc-git: I could either try to fix what's happening (unless it's the symlink issue), or it can be done with a bit of injected Javascript. Either way, if your still interested, please do open a new issue. Love to fix it, but I prefer a clean context for each issue, hence my request to open a new one.
Sure, I think I understand the issue about symlinks. And I was not complaining; sorry if I was not clear. The current behavior I think I understand, and it is fine with me (I do want full paths). I was trying to explain that I can actually get the full path, which I thought was something that @llorracc-git might want (full paths, I mean; regardless of symlinks). Apologies for the noise.