Project dependencies may have API risk issues
PyDeps opened this issue · comments
Hi, In misp-warninglists, inappropriate dependency versioning constraints can cause risks.
Below are the dependencies and version constraints that the project is using
beautifulsoup4
pyOpenSSL==19.1.0
python-dateutil==2.8.1
requests
dnspython
pyasn1
pyasn1-modules
The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict.
The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.
After further analysis, in this project,
The version constraint of dependency beautifulsoup4 can be changed to >=4.10.0,<=4.11.1.
The version constraint of dependency pyOpenSSL can be changed to >=0.14,<=22.0.0.
The version constraint of dependency python-dateutil can be changed to >=2.5.0,<=2.6.1.
The version constraint of dependency requests can be changed to >=0.2.1,<=0.2.3.
The version constraint of dependency requests can be changed to >=0.7.0,<=2.24.0.
The version constraint of dependency requests can be changed to ==2.26.0.
The version constraint of dependency pyasn1 can be changed to >=0.4.1,<=0.4.8.
The version constraint of dependency pyasn1-modules can be changed to >=0.0.3,<=0.2.8.
The above modification suggestions can reduce the dependency conflicts as much as possible,
and introduce the latest version as much as possible without calling Error in the projects.
The invocation of the current project includes all the following methods.
The calling methods from the beautifulsoup4
bs4.BeautifulSoup
The calling methods from the pyOpenSSL
OpenSSL.crypto.load_certificate
The calling methods from the python-dateutil
dateutil.parser.parse
The calling methods from the requests
requests.head requests.get
The calling methods from the pyasn1
open
The calling methods from the pyasn1-modules
pyasn1_modules.rfc2459.AuthorityInfoAccessSyntax pyasn1_modules.rfc2459.CRLDistPointsSyntax
The calling methods from the all methods
p.open cisco_lists.open requests.get lipv6.append file.get_abspath_source_file.path.getmtime.datetime.datetime.fromtimestamp.astimezone a_tag.attrs.get is_akamai crl.getComponentByName unique_sorted_warninglist OpenSSL.crypto.load_certificate.digest get_networks_for_asn site.decode.split.strip bs4.BeautifulSoup logging.getLogger.addHandler ipv6_networks.append generator.download.raise_for_status mx_ips.extend lists.append url.split row.rstrip logging.getLogger.setLevel ip.strip url.find generate_irish_warninglist tranco.readlines map get_domain generate_swedish_warninglist section.getComponentByName tld.startswith get_lists_dnscrypt threads.append lists.sort answer.__iter__.next.to_text url.split.split output.append set.append os.path.exists ipaddress.ip_network urllib.parse.urlparse mobile_numbers.items list.split validate_file lips.append part.startswith bs4.BeautifulSoup.findAll str data_list.append self.get_ip_for_domain section.getComponentByName.getComponentByName addresses.append lhostname.append isinstance json.loads logging.basicConfig v.strip.replace self.resolver.query google_warninglist.append bs4.BeautifulSoup.find generator.download_to_file urllib.request.urlopen re.compile site.split dns.resolver.LRUCache OpenSSL.crypto.load_certificate next line.decode spf_ranges.extend self.openresolvers.append InvalidListValue alexa_lists.open address.split.split os.listdir argparse.ArgumentParser.parse_args is_valid generate_french_warninglist bool validate.extend process generator.Dns.get_ip_ranges_from_spf json.loads.update service.get pyasn1_modules.rfc2459.AuthorityInfoAccessSyntax validate url.replace obj.strip re.findall os.mkdir sorted self.get_ip_ranges_from_spf inspect.stack generate_uk_warninglist netaddr.IPAddress l.append line.startswith cert.get_extension.get_data obj.strip.replace asciidoc sorted.append domain.encode ipaddress.IPv4Address os.path.join site.split.rstrip logging.FileHandler.setFormatter bs4.BeautifulSoup.find_all f.readlines func csv.reader part.split alexa_lists.namelist generator.download.split domain.lstrip actual_download_to_file get_ips_from_domains requests.head gethash int answer.__iter__.next.to_text.strip x.join hostname.split line.strip generator.Dns.get_mx_ips_for_domain csv.DictReader r.text.splitlines multiprocessing.dummy.Pool.map tranco_lists.namelist threading.Thread self.get_mx_ips_for_domain logging.FileHandler get_crl_ocsp_domains generator.create_resolver time.sleep get_abspath_source_file asn_to_fetch.append len generator.consolidate_networks logging.error s.decode lipv4.append codecs.decode self.__resolver.query threading.Thread.start list header os.path.abspath v.rstrip.rstrip self.errors.append dns.resolver.query cert.get_extension.get_short_name urllib.parse.urljoin argparse.ArgumentParser.add_argument argparse.ArgumentParser get_all_website_links pyasn1.codec.der.decoder.decode url.rsplit datetime.datetime.fromtimestamp logging.getLogger requests.get.iter_lines data.startswith site.decode.split.rstrip this.ips.pop digest.cert.digest.decode frame_records.getmodulename.upper logging.warning generator.get_abspath_source_file datetime.datetime.utcnow inspect.getmodulename OpenSSL.crypto.load_certificate.get_extension_count inspect.getframeinfo generate_australian_warninglist.extend list.extend warninglist.append generator.process_stream json.load logging.info OpenResolverChecker.launch format generalName.getComponentByName print dns.resolver.Resolver this.check generator.download.json line.decode.startswith ipv4_networks.append site.decode csv_file.readlines r.headers.parsedate.astimezone row.findAll json.dump this.errors.append re.compile.match ranges.append cisco_list.readlines logging.Formatter quit university.get range sys.exit get_abspath_list_file top.readlines generate_australian_warninglist.append generate_australian_warninglist url.replace.replace urllib.request.urlopen.read ValueError OpenSSL.crypto.load_certificate.get_extension url.download.json init_logging join dateutil.parser.parse ipaddress.ip_address v.rstrip open OpenResolverChecker obj.strip.replace.replace is_valid_regexp pathlib.Path.glob ipaddress.IPv6Address fd.write spf.split logging.exception generator.create_resolver.query row.findAll.find_all cisco_lists.namelist datetime.datetime.now generator.download generate generator.get_version.append set tabl.findAll url.replace.replace.replace _f.write os.path.realpath inspect.currentframe self._parse_spf zipfile.ZipFile set.add search response.read.decode set.update requests.get.iter_content obj.lower get_lists_publidns datetime.date.today multiprocessing.dummy.Pool os.path.dirname retry_link_text.find_all.get generator.write_to_file generate_american_warninglist crl.getComponentByName.getComponentByName tranco_lists.open pathlib.Path pyasn1_modules.rfc2459.CRLDistPointsSyntax get_lists soup.find.find_all lurls.append os.path.getmtime all main domain.startswith datetime.date.today.strftime self.resolver.query.__iter__ get_json_url list.append digest.cert.digest.decode.replace.lower ipaddress.collapse_addresses generator.Dns join.startswith generator.get_version digest.cert.digest.decode.replace address.split.index domain.encode.decode ipaddress.IPv6Interface data_file.write site.decode.split
@developer
Could please help me check this issue?
May I pull a request to fix it?
Thank you very much.
Sure, if you want to make a pull-request. I'm just wondering about the No Upper Bound rule. IMHO, security wise I prefer to not have any upper bound to ensure that people are running the latest version which might include security fixes. API breaking versus security vulnerability in this project, it's clearly the security fixes which are more important are the generator scripts are only to generate the JSON available in this repository.