Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add search functionality #65

Merged
merged 1 commit into from
Jun 24, 2014
Merged

Add search functionality #65

merged 1 commit into from
Jun 24, 2014

Conversation

yannickcr
Copy link
Contributor

I've implemented the search functionality, it should fix #36

Basically, it will fetch package list from npm and simply append the local ones in the response.

This patch:

  • add a new route to match the search queries (/-/all/*).
  • add a new get_recent_packages method in local-storage to list the packages more recent than a given timestamp.
  • add a new search method in storage to run the request to npm and to append the local packages (found with get_recent_packages) to the response.

@rlidwka
Copy link
Owner

rlidwka commented Apr 13, 2014

My biggest concern here is DoS. With this patch anyone who have access to /-/all can crash the process by creating several requests because of buffering.

Does it make sense to include local packages there? Maybe just self.uplinks[uplinks[0]].request(...).pipe(response) would suffice (by which I mean piping output from the first uplink as is without changing it)?

@yannickcr
Copy link
Contributor Author

Sinopia is more likely used on a private network than on a public one so I don't see DoS as a real issue.

We could use the find *nix command in get_recent_packages to directly get the new files list, and not loop around all the files to filter them after. I think it would improve the file listing but sadly this command is not available under Windows. Maybe we can make some OS conditions here.

About the local packages, our needs are to be able to search on the npm registry but also to be able to find the private packages we published on Sinopia.

@danyshaanan
Copy link

I think I'm in a similar situation as @yannickcr . I want to be able to search private packages, (even: want to be able to search only private packages), and since this will most likely be internal, I don't care about DDOS. But if that's a problem in general, it could be only enabled in the config, or even (config.yaml):

search: local_storage storage npmjs

I'd be happy to contribute here if needed.

@phillipj
Copy link

phillipj commented May 7, 2014

+1

Would really appreciate being able to search for packages! Also willing to contribute if needed?

@leftieFriele
Copy link

+1 search would be awesome

@mllrjb
Copy link

mllrjb commented Jun 12, 2014

+1 on searching private. @rlidwka - thoughts?

@ghost
Copy link

ghost commented Jun 12, 2014

+1 on searching private. Not concerned with DoS here either.

@ecowden
Copy link

ecowden commented Jun 17, 2014

Another +1 here. Would looove to see this feature. 👍

rlidwka added a commit that referenced this pull request Jun 24, 2014
@rlidwka rlidwka merged commit 9816059 into rlidwka:master Jun 24, 2014
@jalateras
Copy link

great. I will give this a go

rlidwka added a commit that referenced this pull request Jun 24, 2014
rlidwka added a commit that referenced this pull request Jun 24, 2014
rlidwka added a commit that referenced this pull request Jun 24, 2014
For each of the packages check if user has access to it and remove
package info from the result if he doesn't.

ref #65
@rlidwka
Copy link
Owner

rlidwka commented Jun 24, 2014

Thanks, merging. I also changed it a bit (see commits before), most notably adding access control (with your version anybody who have access to "" package would see anything). Please check that out.

There is still a lot of issues with this, for example it polls only the first uplink (if it's unreachable, returns {} which confuses npm a lot), but I suppose it would work for now.

(sorry for the delayed response, was on vacation for a month)

@yannickcr
Copy link
Contributor Author

Great ! Thanks.

@rlidwka rlidwka mentioned this pull request May 16, 2015
@jmls
Copy link

jmls commented Oct 16, 2015

perhaps I'm misunderstanding something, but I can't get the search to work .

https://registry.npmjs.org/-/_view/byKeyword??startkey=["http"]&endkey=["http",{}]&group_level=2

works fine, however,

http://dev.npm.myrepo/-/_view/byKeyword?startkey=["http"]&endkey=["http",{}]&group_level=2

doesn't

am I missing something, or have I completely misunderstood what the search function does in sinopia ? thanks!

rmg pushed a commit to strongloop-forks/sinopia that referenced this pull request Dec 21, 2016
rmg pushed a commit to strongloop-forks/sinopia that referenced this pull request Dec 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

npm search _something_ doesn't work
9 participants