ffflorian / api-clients

I'm trying to use the https://www.npmjs.com/package/npms.io client to search npms.io by a specific keyword and non-deprecated plugins: keywords:eleventy-plugin not:deprecated

If I was using the API directly, I'd use something like this:
https://api.npms.io/v2/search?q=keywords:eleventy-plugin+not:deprecated

I'm not sure how to do that w/ the NpmsIO client though. I tried this:

import { NpmsIO } from "npms.io";

const npmClient = new NpmsIO();

const plugins = await npmClient.api.search.searchPackage("keywords:eleventy-plugin not:deprecated", {size:250});
console.log(plugins);

… but it always returns zero results: { total: 0, results: [] }

If I remove the encodeURIComponent() from this line and just do q: query in my local ./node_modules/ folder, then I start seeing results:

api-clients/packages/npms.io/src/api/SearchAPI.ts

Line 58 in 72b2161

q: encodeURIComponent(query),

- q: encodeURIComponent(query)
+ q: query

{
  total: 154,
  results: [
    { package: [Object], score: [Object], searchScore: 0.0007207585 },
    { package: [Object], score: [Object], searchScore: 0.00066178717 },
    ...
  ]
}

Also filed upstream at npms-io/npms#53 in case it is an npms.io issue. 🤷

Upstream issue was my fault, it was encoding the + as %2B instead of me using spaces and it encoding as %20.

So the API supports encoded values: https://api.npms.io/v2/search?from=0&q=keywords%3Aeleventy-plugin%20not%3Adeprecated%20scope%3A11ty&size=250

Not sure why I can't get it working through the API:

import { NpmsIO } from "npms.io";

const npmClient = new NpmsIO();

const plugins = await npmClient.api.search.searchPackage("keywords:eleventy-plugin not:deprecated", {size:250});
console.log(plugins); // { total: 0, results: [] }

OUTPUT

// node_modules/npms.io/dist/api/SearchAPI.js
console.log({endpoint, params});

{
  endpoint: '/search',
  params: {
    from: undefined,
    q: 'keywords%3Aeleventy-plugin%20not%3Adeprecated',
    size: 250
  }
}

Hey @pdehaan, I'll look into this ASAP!

Thank you for the detailed report, the trick was indeed to remove encodeURIComponent() and just let axios manage the encoding of the search query.

Awesome, thanks!
Verified latest build works (by building a little script that [non]recursively fetches all the paged results):

import { NpmsIO } from "npms.io";

const npmClient = new NpmsIO();

const plugins = await searchKeywords("keywords:eleventy-plugin not:deprecated");
console.log(plugins);

async function searchKeywords(query = "", size = 250) {
  const packages = [];
  let _total;

  try {
    do {
      console.info(`Fetching ${size} packages (from: ${packages.length})`);
      const { results, total } = await npmClient.api.search.searchPackage(query, {
        size,
        from: packages.length,
      });
      packages.push(...results);
      _total = total;
    } while (packages.length < _total);
  } catch (err) {
    console.error(err.message);
    process.exitCode = 1;
  }
  return packages;
}

https://github.com/pdehaan/npms-client-while

Great, good to see it in action!

Unable to search npms.io using specific keywords and operators

OUTPUT