unexpected end of JSON input

[chapterjason]

Hey,

go: 1.17.2
caddy: 2.4.5

$ xcaddy build v2.4.5 --with github.com/abiosoft/caddy-json-schema
$ ./caddy json-schema -no-cache
json-schema discarding cache for root config.
json-schema fetching https://caddyserver.com/api/docs/config/ ...
json-schema 
json-schema discarding cache for apps/exec.
json-schema fetching https://caddyserver.com/api/docs/config/apps/exec ...
json-schema 
json-schema discarding cache for apps/supervisor.
json-schema fetching https://caddyserver.com/api/docs/config/apps/supervisor ...
json-schema 
json-schema discarding cache for apps/http.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http ...
json-schema 
json-schema discarding cache for apps/pki.
json-schema fetching https://caddyserver.com/api/docs/config/apps/pki ...
json-schema 
json-schema discarding cache for apps/tls.
json-schema fetching https://caddyserver.com/api/docs/config/apps/tls ...
json-schema 
json-schema discarding cache for apps/auth.
json-schema fetching https://caddyserver.com/api/docs/config/apps/auth ...
json-schema 
json-schema: unexpected end of JSON input

The url https://caddyserver.com/api/docs/config/apps/auth response is empty with 502 error code.

Might not be a good idea to use the web server from caddy itself, it's an external services which can always change.
Like in my case it looks like auth is gone?

[Gr33nbl00d]

I can reproduce this, maybe should be reported to caddy project.
Menawhile as a workarround editing the cached json files and remove relevant configs can make it work again

[Em Sharnoff]

Just in case anyone else is coming across this, I found that the following file was sufficient:

{"status_code":200}

These were in <caddy/json_schema/<doc name>/docs.json. At time of writing, the docs that required this were:

• apps/auth
• apps/http.handlers.prometheus
• apps/http.handlers.request_debug
• apps/http.handlers.image_filter

[Francis Lavoie]

This should now be fixed, see caddyserver/website#206. Could any of you give this a try again?

[Mike Degatano]

@francislavoie just hit this, from a different endpoint though. I see it from apps/http.handlers.prometheus:

json-schema cached docs not found for apps/http.handlers.prometheus.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http.handlers.prometheus ...
json-schema
json-schema: unexpected end of JSON input

[Francis Lavoie]

I don't think that plugin was registered on the Caddy website, so there's no docs for it. But either way, metrics are built into Caddy now, so that plugin shouldn't be necessary 🤔

[Mike Degatano]

Ok I get it, there's a race condition to see which one breaks first. So of the ones sharnoff listed above endpoints for the following 3 were returning 502 and were stopping json schema generation:

• apps/http.handlers.prometheus
• apps/http.handlers.request_debug
• apps/http.handlers.image_filter

After manually putting json files into the cache for those 3 generation completed susccessfully. I did not have to touch anything for apps/auth so it looks like that one is fixed.

I'm not really sure why its pulling in prometheus if it shouldn't @francislavoie . I don't use prometheus so I'm not building in an extension around it with xcaddy. Perhaps its on a list and needs to be removed? All 3 of those modules say "non-standard" in the docs so seems like they're being incorrectly included.

[Francis Lavoie]

The "non-standard" label means it's not included with vanilla Caddy, i.e. a third party plugin.

Also, I was wrong that they aren't registered on the website, they are listed here: https://caddyserver.com/docs/modules/ but I don't see which Go module they come from 🤔

I think @mholt will need to take a look at why those are causing issues. I don't have direct access to the website backend.

[Matt Holt]

That's weird. Looks like something is wrong with the site for those docs (JS errors) -- sigh, sorry about that. I will have to look into that as soon as I have a chance. (And I've noticed some docs where the yellow box explaining where the non-standard module's repo can be found is missing... hmm.)

[Matt Holt]

Ok, I think it's just that the auth plugin was moved/renamed to security: https://caddyserver.com/docs/modules/security

And that does seem to work fine.

[AlexanderAllen]

Running into this when building docs for https://github.com/mholt/caddy-l4.

Dockerfile

RUN \
    apk update \
    && apk add $run_deps \
    && go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest \
    \
    # https://caddyserver.com/docs/modules/layer4
    # https://caddyserver.com/docs/build
    && /root/go/bin/xcaddy build ${CADDY_VERSION} \
        --with github.com/abiosoft/caddy-json-schema \
        --with github.com/mholt/caddy-l4/layer4 \
    \
    && mv caddy /usr/local/bin

Build command:

 # caddy json-schema --vscode --output /out/caddy_schema.json

Output:

json-schema cached docs not found for root config.
json-schema fetching https://caddyserver.com/api/docs/config/ ...
json-schema
json-schema cached docs not found for apps/exec.
json-schema fetching https://caddyserver.com/api/docs/config/apps/exec ...
json-schema
json-schema cached docs not found for apps/supervisor.
json-schema fetching https://caddyserver.com/api/docs/config/apps/supervisor ...
json-schema
json-schema cached docs not found for apps/http.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http ...
json-schema
json-schema cached docs not found for apps/pki.
json-schema fetching https://caddyserver.com/api/docs/config/apps/pki ...
json-schema
json-schema cached docs not found for apps/tls.
json-schema fetching https://caddyserver.com/api/docs/config/apps/tls ...
json-schema
json-schema cached docs not found for apps/security.
json-schema fetching https://caddyserver.com/api/docs/config/apps/security ...
json-schema
json-schema cached docs not found for apps/crowdsec.
json-schema fetching https://caddyserver.com/api/docs/config/apps/crowdsec ...
json-schema
json-schema cached docs not found for apps/dynamic_dns.
json-schema fetching https://caddyserver.com/api/docs/config/apps/dynamic_dns ...
json-schema
json-schema cached docs not found for apps/layer4.
json-schema fetching https://caddyserver.com/api/docs/config/apps/layer4 ...
json-schema
json-schema cached docs not found for apps/layer4.matchers.crowdsec.
json-schema fetching https://caddyserver.com/api/docs/config/apps/layer4.matchers.crowdsec ...
json-schema
json-schema cached docs not found for apps/caddy.logging.writers.stdout.
json-schema fetching https://caddyserver.com/api/docs/config/apps/caddy.logging.writers.stdout ...
json-schema
json-schema cached docs not found for apps/http.matchers.exec_noop.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http.matchers.exec_noop ...
json-schema
json-schema cached docs not found for apps/http.matchers.execnopmatch.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http.matchers.execnopmatch ...
json-schema
json-schema cached docs not found for apps/http.handlers.geofence.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http.handlers.geofence ...
json-schema
json-schema cached docs not found for apps/dns.providers.lego_deprecated.
json-schema fetching https://caddyserver.com/api/docs/config/apps/dns.providers.lego_deprecated ...
json-schema
json-schema cached docs not found for apps/caddy.logging.encoders.filter.
json-schema fetching https://caddyserver.com/api/docs/config/apps/caddy.logging.encoders.filter ...
json-schema
json-schema cached docs not found for apps/http.handlers.request_debug.
json-schema fetching https://caddyserver.com/api/docs/config/apps/http.handlers.request_debug ...
json-schema
json-schema: unexpected end of JSON input

I'm not a Go or Caddy expert, so maybe I'm doing something wrong?

[Matt Holt]

@AlexanderAllen Those paths look broken, starting with /api/docs/config/apps/layer4.matchers.crowdsec -- there is no app named layer4.matchers.crowdsec; the correct link should probably be https://caddyserver.com/api/docs/module/http.handlers.request_debug, but this is different than config-traversal docs, if that makes sense. I'm not sure what is generating those paths but it seems to be an error.

(That particular module does not appear to have docs btw. There's JSON response but the docs are null. Hmm.)

[AlexanderAllen]

Hmm, there's probably something wrong with my Docker build. I'm also using Alpine, and usually that's an "all bets are off" for most people when it comes to compiling anything. Cheers for the response.

…

On Mon, Mar 28, 2022 at 12:04 AM Matt Holt ***@***.***> wrote: @AlexanderAllen <https://github.com/AlexanderAllen> Those paths look broken, starting with /api/docs/config/apps/layer4.matchers.crowdsec -- there is no app named layer4.matchers.crowdsec; the correct link should probably be https://caddyserver.com/api/docs/module/http.handlers.request_debug, but this is different than config-traversal docs, if that makes sense. I'm not sure what is generating those paths but it seems to be an error. (That particular module does not appear to have docs btw. There's JSON response but the docs are null. Hmm.) — Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADK6SRMW5X7EBDBHM2B3RF3VCEVVFANCNFSM5HWKW2OQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Richard Allen Mobile: [phone]

[Francis Lavoie]

We ship a caddy:builder image variant that you can use (see https://hub.docker.com/_/caddy) which is also based on Alpine. It's definitely not an issue with your build.

There's definitely a bug in caddy-json-schema that needs to be fixed, it's generating incorrect URLs for trying to fetch each module's docs.

[AlexanderAllen]

Hmm. Makes me want to jump in and debug my current use case, whatever that is, since it looks from the comments it's not replicated across the board. Could it be one of the dependencies getting pulled in during build causing unexpected issues (works for you but not me scenario, different "lockfiles"), but that's a tough one to confirm. I'm not a Go supremo exceptionale, but this sounds like a good candidate to burn 8 hours on the weekend :)

…

On Tue, Mar 29, 2022 at 1:44 AM Francis Lavoie ***@***.***> wrote: We ship a caddy:builder image variant that you can use (see https://hub.docker.com/_/caddy) which is also based on Alpine. It's definitely not an issue with your build. There's definitely a bug in caddy-json-schema that needs to be fixed, it's generating incorrect URLs for trying to fetch each module's docs. — Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADK6SRMPNK3V7XRII3JP22LVCKKE3ANCNFSM5HWKW2OQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Richard Allen Mobile: [phone]

[Malcolm Crum]

I reproduced the issue:

docker run --rm -it caddy:builder sh
xcaddy build v2.4.6 --with github.com/abiosoft/caddy-json-schema
./caddy json-schema

Error is json-schema: unexpected end of JSON input

[Fedor Golishevskii]

Fixed in #8
You can test it out using my fork https://github.com/fedorg/caddy-json-schema
It contains build instructions for Docker and a gist with the generated schema file.

[Mike Degatano]

Was going to note that I was still hitting this when generating the schema for 2.5.0 but I see now that there's a PR to fix it which is great. If anyone is keeping track of which plugins have this doc issue, I hit it for all 3 of the ones I mentioned before plus one new one. Here's the current list of the plugins I had to manually fix docs.json for:

• apps/http.handlers.prometheus
• apps/http.handlers.request_debug
• apps/http.handlers.image_filter
• apps/http.handlers.olaf

[Matt Holt]

Thanks @mdegat01 -- quite likely a bug in the docs system I wrote. I'm not sure what those have in common yet, though, so I'll need to find some time to take a look and see what's going on.