QST: Has the finviz page changed since 28 sept 2023?

[ljilekor]

Have there been changes to the finviz.com tables since 28 sept 2023?

[ividrevich]

im having issues running finviz screeners today using the library. Everything was working as of yesterday.

Heres Traceback report:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [4], in <cell line: 4>()
      2 results = pd.DataFrame()
      3 foverview.set_filter(filters_dict=filters, ticker="", signal=SIGNALS[0])
----> 4 df = foverview.screener_view(verbose=0)
      5 df = df[['Ticker']]
      6 df['Pattern'] = SIGNALS[0]

File ~/opt/anaconda3/envs/mm-strat/lib/python3.9/site-packages/finvizfinance/screener/overview.py:269, in Overview.screener_view(self, order, limit, select_page, verbose, ascend)
    266         progress_bar(1, 1)
    268 table = soup.find("table", class_="table-light")
--> 269 rows = table.findAll("tr")
    270 table_header = [i.text for i in rows[0].findAll("td")][1:]
    271 num_col_index = [table_header.index(i) for i in table_header if i in NUMBER_COL]

AttributeError: 'NoneType' object has no attribute 'findAll'

[jdejones]

AttributeError Traceback (most recent call last)
1 overview.set_filter(filters_dict={'Earnings Date': 'Today After Market Close'})
----> 2 overview.screener_view(order='Market Cap.')

278 
279         table = soup.find("table", class_="table-light")

--> 280 rows = table.findAll("tr")
281 table_header = [i.text.strip() for i in rows[0].findAll("td")][1:]
282 num_col_index = [table_header.index(i) for i in table_header if i in NUMBER_COL]

AttributeError: 'NoneType' object has no attribute 'findAll'

I have the same error. I'm using finvizfinance version 0.14.6.

[Pengchong Tang]

Same here, look like sucked by Finviz.

[Matteo Seminaroti]

any chance this issue will be solved soon? I have the same error since yesterday

[James Maslek]

For anyone looking for a workaround, I got this to monkeypatch the screener_view function. But you would need to overwrite it in every class (and see what the relevant args are). I also do not promise it is stable, but it works until the authors dig in:

from finvizfinance.screener.overview import Overview # THIS SHOULD BE THE PATH TO THE OVERVIEW YOU ARE USING
from finvizfinance.util import web_scrap

# It works but it's not pretty
def screener_view(self, group="Sector", order="Name"):
    """Get screener table.
    Args:
        group(str): choice of group option.
        order(str): sort the table by the choice of order.
    Returns:
        df(pandas.DataFrame): group information table.
    """
    if group not in self.group_dict:
        group_keys = list(self.group_dict.keys())
        raise ValueError(
            f"Invalid group parameter '{group}'. Possible parameter input: {group_keys}"
        )
    if order not in self.order_dict:
        order_keys = list(self.order_dict.keys())
        raise ValueError(
            f"Invalid order parameter '{order}'. Possible parameter input: {order_keys}"
        )
    self.url = (
        self.BASE_URL.format(group=self.group_dict[group], v_page=self.v_page)
        + "&"
        + self.order_dict[order]
    )

    soup = web_scrap(self.url)
    return pd.read_html(str(soup))[-2]


Overview.screener_view = screener_view

edit: only works on the group.overview.

[Abdul Haseeb Khan]

I'm having the same issue in overview.py line 295 296
table = soup.find("table", class_="table-light")
rows = table.findAll("tr")

Exception caught: 'NoneType' object has no attribute 'findAll'

Any fix for this?

Here's how I'm using it

overview = Overview()
overview.set_filter(filters_dict=filters_dict)
df = overview.screener_view(order='Volume', ascend=False)

Thanks.

[Tianning Li]

I will fix this. Thanks for reporting.

[Tianning Li]

Version 0.14.7rc1 is published. Please check.

[Abdul Haseeb Khan]

Thanks a ton!
Just checked, used a variety of filters and it worked.

[ljilekor]

Great to see this got fixed quickly after I posted this issue.
Finviz changed a few things, so I fixed some stuff and got it working(with the previous version) and continued working on my custom version.
I only changed the 'Custom' screener

Mainly changed this

        # since 28 sep 2023
        #table = soup.find("table", class_="table-light")        
        table = soup.find("table", class_="styled-table-new is-rounded is-tabular-nums w-full screener_table")
        ...
        #table_header = [i.text.strip() for i in rows[0].findAll("td")][1:]
        table_header = [i.text.strip() for i in rows[0].findAll("th")][1:]

Maybe some other stuff...

I only changed it in the 'Custom' screener.
@lit26 Are all the modules fixed? (Groups, Forex, Futures, News, ...)
Do you have an overview of what was changed?

[Tianning Li]

@ljilekor I have updated most of the parsing in screener, group and others. The files changes is here. https://github.com/lit26/finvizfinance/pull/84/files#diff-55c34951e474479bab690402d7fc126b898dc2d640382dcef2c62a55e10c04c1