Errors with external content.sources ?
mathieunicolas opened this issue · comments
Hi Dave ! I come from here nuxt/content#1837
Describe the bug
I use content.sources
in nuxt.config.ts to fetch a repo where there's MD files and some media files.
I tried your module, and I have this error repeatedly, and a 404 error when I go to :3000 :
[nuxt] [request error] [unhandled] [500] The "path" argument must be of type string. Received undefined 23:02:35
at new NodeError (node:internal/errors:399:5)
at validateString (node:internal/validators:163:11)
at Object.join (node:path:1176:7)
at getDocPath (./.nuxt/dev/index.mjs:598:15)
at ./.nuxt/dev/index.mjs:607:22
at ./node_modules/hookable/dist/index.mjs:48:66
at ./node_modules/hookable/dist/index.mjs:48:56
at async parseContent (./.nuxt/dev/index.mjs:3574:3)
at async getContent (./.nuxt/dev/index.mjs:3550:18)
at async Promise.all (index 1)
To Reproduce
add this to a docus template and install nuxt-content-assets :
content: {
sources: {
gh: {
driver: 'github',
repo: 'mathieunicolas/alloe',
prefix: '/'
}
},
navigation: {
fields: ['auteur']
}
},
Additional context
Add any other context about the problem here
Software versions:
Run the code below and replace with the output:
Node v19.6.0
docus-starter@0.1.0 /Users/mathieu/WebstormProjects/alloe-site
├── nuxt-content-assets@0.6.1
└── nuxt@3.3.3
Does your module handle external sources ? Everything works fine if I manually copy the content of the repo in my content
folder.
Cheers from France !
Hello!
Thanks for reporting your use case and the bug.
Right now, no, it doesn't support external sources, but only because I haven't used them, so don't know anything about them (and had assumed they wouldn't work).
If I point the demo at your repo I'll be able to run it?
If so I can dig in and see if it's possible to properly integrate.
I'll report back!
Okay thanks, I thought it would work since you posted your module in an issue where I was talking about external sources ^^
I will continue developing my project by manually pasting the external repo in the content
folder.
When I posted the issue, my code worked. But now it doesn't work anymore, so I'm afraid I won't be able to help in the next 10 days. We can look after that if you want !
I paste here the nitro plugin that worked few months ago :
import fs from "fs";
export default defineNitroPlugin((nitroApp) => {
nitroApp.hooks.hook('content:file:beforeParse', async (file) => {
const extension = file._id.split('.')
const statics = ['png','jpg','pdf','epub', 'excalidraw']
if (statics.includes(extension.pop())) {
const filepath_arr = file._id.split(':') // array of driver name, then every subfolder, then filename
const filepath = './public/'+filepath_arr.slice(1).join('/') // contains the full path to the filename
const filedir = './public/'+filepath_arr.slice(1, -1).join('/') // the folder to create because it contains the file
fs.mkdirSync(filedir, { recursive: true })
if(file.body.constructor.name === 'Blob') {
const imageBuffer = await file.body.arrayBuffer();
fs.createWriteStream(filepath).write(Buffer.from(imageBuffer))
} else if (true) { // some space to be more precise if needed, idk how
fs.createWriteStream(filepath).write(JSON.stringify(file.body, undefined, 2))
}
}
})
})
PS : the code was a dirty poc, waiting for refining !
Don't worry, I figured it!
No freaking idea how to intercept the paths generated by Nuxt Content though.
I know it gets them using unstorage, but no idea how or where I am supposed to hook into this functionality.
Hehey... a little progress in setting up the storage and grabbing the images from the repo...
[
'gh:alloe18:3arR5cP.jpg',
'gh:alloe18:machine.png',
'gh:alloe18:pommier.png',
'gh:alloe18:puit.png',
'gh:alloe18:reparation.png',
'gh:alloe18:rouages.png',
'gh:alloe18:tonneaux.png',
'gh:alloe19:JzU6Iyo.jpg',
'gh:alloe20:Zx13oLOe.png',
'gh:alloe21:468zwUH1.png',
'gh:logo-alloe.excalidraw',
'gh:public:alloe2018.epub',
'gh:public:alloe2018.pdf',
'gh:public:alloe2019.epub',
'gh:public:alloe2019.pdf',
'gh:public:alloe2020.epub',
'gh:public:alloe2020.pdf',
'gh:public:alloe2021.epub',
'gh:public:alloe2021.pdf'
]
And it looks like I can get these as Blobs, so maybe can use your code above...
[
Blob {},
Blob {},
Blob {},
...
I don't have a lot of time to spare at the moment, but 🤞 for a quick win.
Woaaaaw you're so quick !!
Btw, I used your module with local content files, and it's amazing, thanks for sharing such a useful tool !!
Rats; thought I had it all working, but no.
I won't be able to test until monday, congrats on what you've accomplished ! What's not working ?
It works perfectly in dev, but in static generation the images aren't fetched so the copy step breaks.
I was out last night but I'm going to look at it first thing today.
OK! As of version 0.7.0 Nuxt Content Assets now supports GitHub external sources 🚀
Congratulations @davestewart for this ultra quick feature add ! It's amazing.
I have a problem with SRC rewriting on IMG tags :
https://64331db4db504a69ee36d552--kaleidoscopic-brioche-dc7d6b.netlify.app/alloe18/partie1/chapitre1/
you can find here the generated page for this file
my nuxt.config.ts
:
'content-assets': {
output: '/[folder]/[file]',
additionalExtensions: 'epub',
},
The file machine.png
is inserted in the /alloe18/partie1/chapitre1.md
file with the code ![image](../machine.png)
,
BUT it is copied in the path /alloe18/machine.png
AND is referenced as alloe18/partie1/alloe18/machine.png
in the generated
I guess alloe18/partie1
is the ../machine.png
from the MD, and alloe18/machine.png
is the correct path of the file, so maybe there's a concatenation where there should be a replacement of the image URL string ?
Again, thanks and congratz !
The issue is closed so I don't know if you will be notified for this message, let me know !
Actually @mathieunicolas – can you create this as a new issue please?
🙏
PS. The 0.8.0 version has done away with the extensions configuration. Anything now that is not a markdown file is presumed to be an assset!
Question: Does the external Github source work dynamically? In other words, is it a requirement of the assets module to re-build the application in order to serve static assets from an external source?
Ideally what I want is for the application to build once and automatically reflect changes to external sources. I just wrote a webdav driver for unstorage, and my goal is for authors to be able to simply write content on their own desktop or online via NextCloud. This currently works with Nuxt Content module, although I needed to submit a PR for Nuxt Content to support building custom drivers.
I am interested in helping out with Nuxt Content Assets module to make this possible.
Hey @shareable-vision,
Sorry, only just saw this.
Short answer I think is "no".
Caveat: I'm not really a Nuxt expert, more of a "build what I think I need" kind of plugin developer.
As I understand it, Nuxt Content is a build-time operation; it generates cached files and these are served by Content's API.
Nuxt Content Assets runs immediately after this initial caching, detects relative images, then copies images from content to public, and finally rewrites paths as absolute paths, as outlined here.
I actually don't know enough about Nuxt's other rendering modes to know what the options would be.
TBH, it took a fair bit of investigation and probably a bit of luck to get NCA working and I'm slightly hesitant to unpick it all but it's OSS of course, so more than happy for you to have a look into it 🙂
Thanks for the response, @davestewart.
After running the gamut, I am making the conclusion that this kind of behavior is ubiquitous across all front-end frameworks (take dynamic sources and incessantly re-build them to static bundles).
I am focusing my work on making publishing simple for authors and not for developers per se with as little re-building and deployment operations as possible.
Thanks for your awesome project! There's no way for Nuxt Content to be viable without being smart enough to bundle the assets . . The Nuxt team should take notice of your work and attempt to build a similar solution into their main module.
As for me, I am bowing out of front-end framework universe to get closer to the DOM and the client runtime.
@shareable-vision - I don't blame you!
TBF I think even projects like Astro don't seem to intuitively support relative paths which is a surprise to me.
It just seems something obvious that it should be supported at the expense of more background gymnastics.
Anyway. Good luck with your endeavours!
:D