omer-metin / TweetCollector

Collect Tweets

Tweet Collector

Tweet Collection script that runs with selenium. Collect tweets of tags (#apple), stocks ($AAPL) and words (apple)

To collect large amount of tweet without struggling with Twitter, requires twitter account. (A verified one is strongly recommended.)

Usage

```bash
    python tweet_collector.py --options
```

Options:

• h, --help

• -k, --searchKey KEY (required) search key will be searched

• -a, --search_as [tag, stock, word] (default: tag) search method tag -> $, stock -> $ and word is direct word search

• -s, --start_date START_DATE (required) starting date for search in YYYY-MM-DD format

• -e, --end_date END_DATE (required) final date for search in YYYY-MM-DD format

• -l, --lang SHORTHAND_LANG (default: en) language of tweet to be collected. See supported languages

• -f, --settings_file BOOL (required) (default: True) use settings.json file (ignore setting parameters)

• -u, --username USERNAME
  username of twitter account

• -p, --password PASSWORD
  password of twitter account

• -d, --chromedriver_path CHROMEDRIVER_PATH (default: chromedriver.exe) chromedriver path that is used

• -t, --thread_count THREAD_COUNT (default: 0) number of thread that is used in program

• -m, --missing_run_count MISSING_RUN_COUNT (default: 1) re-run number for missings dates

Examples

• Using settings.json file

  Example settings file

  {
      "username": "*******",
      "password": "*******",
      "chromedriver_path": "chromedriver.exe",
      "thread_count": 0,
      "missing_run_count": 1
  }

  Running code

  python tweet_collector.py -k AAPL -a stock -s 2020-07-28 -e 2020-08-28 -f True

• Without settings.json file

  Running code
  ```bash
  python tweet_collector.py -k AAPL -a stock -s 2020-07-28 -e 2020-08-28 -f False -u ******* -p *******
  

Requirements

• Python 3.6+

  written in 3.7.9

• Selenium

  You can install by using pip

• chromedriver

  You can get proper version from chromedriver

License

MIT