This project basically implements a cherrypy webserver to search and list stock data. It downloads the data from a given url, parses it stores it in a redis database. When user search a stock it looks into the database fetches it and returns the redered html as a table structure.
|-- LICENSE
|-- README.md
|-- requirements.txt
|-- main.py
`-- public
|-- css
| `-- style.css
`-- index.html
-
get_download_url(url)
: It takes the original givenurl
as argument fetch the webpage extracts all the links in the page then filters the link referred to bhav copy zip file. It returns the download link. -
download_zip_file(download_url)
: It takes thedownload_url
as argument downloads and saves it in working directory and returns ZIP file name. -
extract_zip(zip_file_name)
: Takes thezip_file_name
as argument and extracts the CSV file in it and returns the CSV file name. -
prepare_csv_data(csv_file_name)
: Takes thecsv_file_name
as argument reads the csv file prepares the data and returns a list of dictionary which holds the data. The return type is[{key1:'val11',..},{key1:'val12',..},...{key1:'val1n',..}]
. -
strip_name(data_dict)
: Basically theSC_NAME
field contains the name of each Stock which has more than one empty spaces at last. This function takes an individualdata_dict
as argument then gets the value of the key'SC_NAME'
then strips the empty space and returns the dictionary. -
modify_name_field(bhav_copy_dict)
: Here it takes the entire list of data dictionary updates the'SC_NAME'
field by the help ofstrip_name()
function and returns it. -
stock_names(bhav_copy_dict)
: Creates a list of all stock names from thebhav_copy_dict
passed, ehich will be used for setting hash keys and searching purposes. -
prepare_database(pipeline, bhav_copy_dict)
: It takes anpipeline
object and the list of dictionary as input sets the data in redis database server. -
prepare_template(template_dir)
: This function takes thetemplate_dir
as arguments where all the static files are stored and creates a template object ofindex.html
and returns it. -
auto_complete_list(search_str, names)
: Takes thesearch_str
and list of stock names as input, search for starting pattern in each item of the list. If any match or matches found returns the append it to a list and returns it. -
get_from_db(key)
: Heretype(url) = str
. Takes thehash key
as the arguments fetches the data from database and returns it. It also uses a connection object which is globally availavle. -
DisplayStocks()
: This is the main class. A instance of this will be provided as a argument incherrypy.quickstart()
function. It has two main methods explained below.index(count=10)
: This method controls the logic how the home page will be displayed.Thecount
argument passed to show the number of stocks data to be rendered. It is controlled by serverside logic. Need to implement client side logic to control the number of stocks to be displayed.search_stock(name='')
: It takes a 'string' as an user argument from client side and search through thestock_names
list, if found any matches retrieves the data from database renders the html and returns it to client.
Basically it renders the index.html
document in public/
folder. The template.render()
function takes 3 arguments to render a dynamic html document. First argument is a fields
list, the second is count
and the third is the list of data in dictionary format named stock_data
.
<tr>
{% for field in fields %}
<th>{{field}}</th>
{% endfor %}
</tr>
Here in the above first we render the table header with required fields. For this reason we use a for-loop
block. For this we need fields
as first argument passed in the template.render()
function.
{% for data in stock_data[:count] %}
<tr>
{% for field in fields %}
<td>{{data[field]}}</td>
{% endfor %}
</tr>
{% endfor %}
In the above case we need three arguments. First is the stock_data
and the count
number. The stock_data
is a list
of dictionary
object which holds the data. The fields
argument is the same as above.
The first for loop
iterates through the each item in stock_data[:count]
and then the second for loop
iterates through the each field
in fields
and renders the html table.
Explanation of template.render()
template.render(fields=FIELDS, count=cnt, stock_data=list(map(lambda x: get_from_db(x), search_list[:cnt])))
This function takes three arguments already mentioned above.
get_from_db(x)
function takes the hash key value as the argument
retrieve data from database. Here in this function we also use a global client named conn
.
search_list
is the list
object contains the value of SC_NAME
fields. The map()
function maps each item of search_list to get_from_db()
function and returns a python generator
object. Then use the list()
function to iterate through each mapping and retrieves the data.