scholarly
scholarly is a module that allows you to retrieve author and publication information from Google Scholar in a friendly, Pythonic way.
Usage
Because scholarly
does not use an official API, no key is required.
Simply:
import scholarly
print(next(scholarly.search_author('Steven A. Cholewiak')))
Methods
search_author
-- Search for an author by name and return a generator of Author objects.
>>> search_query = scholarly.search_author('Manish Singh, Rutgers')
>>> print(next(search_query))
{'_filled': False,
'affiliation': 'Rutgers University, New Brunswick, NJ',
'citedby': 2906,
'email': '@ruccs.rutgers.edu',
'id': '9XRvM88AAAAJ',
'interests': ['Human perception', 'Computational Vision', 'Cognitive Science'],
'name': 'Manish Singh',
'url_picture': '/citations/images/avatar_scholar_150.jpg'}
search_keyword
-- Search by keyword and return a generator of Author objects.
>>> search_query = scholarly.search_keyword('Haptics')
>>> print(next(search_query))
{'_filled': False,
'affiliation': 'Stanford University',
'citedby': 22427,
'email': '@cs.stanford.edu',
'id': '4arkOLcAAAAJ',
'interests': ['Robotics', 'Haptics', 'Human Motion'],
'name': 'Oussama Khatib',
'url_picture': '/citations/images/avatar_scholar_150.jpg'}
search_pubs_query
-- Search for articles/publications and return generator of Publication objects.
>>> search_query = scholarly.search_pubs_query('Perception of physical stability and center of mass of 3D objects')
>>> print(next(search_query))
{'_filled': False,
'bib': {'abstract': 'Humans can judge from vision alone whether an object is '
'physically stable or not. Such judgments allow observers '
'to predict the physical behavior of objects, and hence '
'to guide their motor actions. We investigated the visual '
'estimation of physical stability of 3-D ...',
'author': 'SA Cholewiak and RW Fleming and M Singh',
'eprint': 'https://scholar.google.comhttp://jov.arvojournals.org/article.aspx?articleid=2213254',
'title': 'Perception of physical stability and center of mass of 3-D '
'objects',
'url': 'http://jov.arvojournals.org/article.aspx?articleid=2213254'},
'citedby': 7,
'id_scholarcitedby': '15736880631888070187',
'source': 'scholar',
'url_scholarbib': 'https://scholar.googleusercontent.com/scholar.bib?q=info:K8ZpoI6hZNoJ:scholar.google.com/&output=citation&scisig=AAGBfm0AAAAAWC-SoOcBZW3iN5qdBiCjkGlFJXqEZVgS&scisf=4&ct=citation&cd=0&hl=en'}
Example
Here's a quick example demonstrating how to retrieve an author's profile then retrieve the titles of the papers that cite his most popular (cited) paper.
>>> # Retrieve the author's data, fill-in, and print
>>> search_query = scholarly.search_author('Steven A Cholewiak')
>>> author = next(search_query).fill()
>>> print(author)
>>> # Print the titles of the author's publications
>>> print([pub.bib['title'] for pub in author.publications])
>>> # Take a closer look at the first publication
>>> pub = author.publications[0].fill()
>>> print(pub)
>>> # Which papers cited that publication?
>>> print([citation.bib['title'] for citation in pub.get_citedby()])
Installation
Use pip
to install from pypi:
pip install scholarly
or pip
to install from github:
pip install git+https://github.com/OrganicIrradiation/scholarly.git
or clone the package using git:
git clone https://github.com/OrganicIrradiation/scholarly.git
Requirements
Requires arrow, Beautiful Soup, bibtexparser, and requests[security].
Changes
Note that because of the nature of web scraping, this project will be in perpetual alpha.
v0.2.2
- Minor fixes to get core running again. Needs more tests. Captcha handling not currently operational.
v0.2.1
- Renamed Publication function citedby() to get_citedby(). New Publication attribute citedby, which just gives the number of citations an article has. Also updated test.py.
v0.2
- Python 2/3 compatibility. No longer using datetime-util and moved the datetime operations to arrow. Now using wheel format.
v0.1.5
- Exactly the same as v0.1.5, but had to bump the version because of a version mistakenly pushed to pypi that had a bad tarball url.
v0.1.4
- Moved over to requests. When Google requests a CAPTCHA, print a URL to the image (rehosted on postimage.org), and have the user confirm that this is being run interactively. Also explicitly request the 'html.parser' for BeautifulSoup. Includes a few small updates to test.py tests to account for updated citation contents and updates to the README. And finally, the pypi install should also now include requests[security].
v0.1.3
- Raise an exception when we receive a Bot Check. Reorganized test.py alphabetically and updated its test cases. Reorganized README. Added python-dateutil as installation requirement, for some reason it was accidentally omitted.
v0.1.2
- Now request HTTPS connection rather than HTTP and update test.py to account for a new "Zucker". Also added information for the v0.1.1 revision.
v0.1.1
- Fixed an issue with multi-page Author results, author entries with no citations (which are rare, but do occur), and added some tests using unittest.
v0.1
- Initial release.
License
The original code that this project was forked from was released by Bello Chalmers under a WTFPL license. In keeping with this mentality, all code is released under the Unlicense.