parse_metar_file with file object not working

[Barry Martin]

What went wrong?

Trying to parse a metar file using a file object that is returned from the remote_open method on a siphon.catalog.Dataset object. According to the docs, the filename parameter can be an actual filename (string) or a file object. The result is a TypeError.

Operating System

MacOS

Version

1.6.1

Python Version

3.12.2

Code to Reproduce

import os
from datetime import datetime

from metpy.io import parse_metar_file
from siphon.catalog import TDSCatalog

cat = TDSCatalog("https://thredds.ucar.edu/thredds/catalog/noaaport/text/metar/catalog.xml")
date = datetime.utcnow()
ds = cat.datasets.filter_time_nearest(date)

## DOES NOT WORK
file_obj = ds.remote_open()
data = parse_metar_file(file_obj)

## WORKS
# ds.download()
# data = parse_metar_file(ds.name)
# os.remove(ds.name)

data.head()

Errors, Traceback, and Logs

TypeError                                 Traceback (most recent call last)
Cell In[202], line 13
     11 ## DOES NOT WORK
     12 file_obj = ds.remote_open()
---> 13 data = parse_metar_file(file_obj)
     15 ## WORKS
     16 # ds.download()
     17 # data = parse_metar_file(ds.name)
     18 # os.remove(ds.name)
     20 data.head()

File ~/opt/anaconda3/envs/metpy/lib/python3.12/site-packages/metpy/io/metar.py:383, in parse_metar_file(filename, year, month)
    379 # Open the file
    380 with contextlib.closing(open_as_needed(filename, 'rt')) as myfile:
    381     # Merge multi-line METARs into a single report--drop reports that are too short to
    382     # be a METAR with a robust amount of data.
--> 383     return _metars_to_dataframe(filter(lambda m: len(m) > 25, full_metars(myfile)),
    384                                 year=year, month=month)

File ~/opt/anaconda3/envs/metpy/lib/python3.12/site-packages/metpy/io/metar.py:441, in _metars_to_dataframe(metar_iter, year, month)
    439 # Try to parse each METAR that is given
    440 metars = []
--> 441 for metar in metar_iter:
    442     with contextlib.suppress(ParseError):
    443         # Parse the string of text and assign to values within the named tuple
    444         metars.append(parse_metar(metar, year=year, month=month))

File ~/opt/anaconda3/envs/metpy/lib/python3.12/site-packages/metpy/io/metar.py:370, in parse_metar_file.<locals>.full_metars(x, prefix)
    368     continue
    369 # No prefix signals a new report, so yield
--> 370 if not i.startswith(prefix) and tmp:
    371     yield ' '.join(tmp)
    372     tmp = []

TypeError: startswith first arg must be bytes or a tuple of bytes, not str

[Ryan May]

remote_open() defaults to opening the data as a binary, but the METAR parser needs things opened as text. Try adjusting your call to remote_open() as:

file_obj = ds.remote_open(mode='t')

[Barry Martin]

Yes it works, I apologize

[Ryan May]

No worries. We could do a better job documenting that requirement for file-like objects.