DHI / terracotta

Do you see any errors in the server log?

In flask application there are no errors. The endpoint is serving image properly. However i am using gunicorn and nginx to serve the application. Whenever i increase the worker count i am not getting this issue.

What's your setup? Assuming the server is running on a single VM? Where are the images (local disk, S3, ...)?

I am running in Azure VM with 4 CPU core, RAM 16 GB and with 128 GB SSD for storage , Yes the tifs bands are in my local storage.

Looks like one of the bands is failing or gets an unexpected offset.

Whenever i increase the worker count i am not getting this issue

That is strange. Is that behaviour consistent? And at which worker size?

And does it change when you disable multiprocessing with USE_MULTIPROCESSING=false?

Thinking about this part here - reading each band in its own process...

terracotta/terracotta/drivers/geotiff_raster_store.py

Lines 29 to 48 in 8632e1c

    
           def create_executor() -> Executor: 
        
               settings = get_settings() 
        
               if not settings.USE_MULTIPROCESSING: 
        
                   return ThreadPoolExecutor(max_workers=1) 
        
               executor: Executor 
        
               try: 
        
                   # this fails on architectures without /dev/shm 
        
                   executor = ProcessPoolExecutor(max_workers=3) 
        
               except OSError: 
        
                   # fall back to serial evaluation 
        
                   warnings.warn( 
        
                       "Multiprocessing is not available on this system. " 
        
                       "Falling back to serial execution." 
        
                   ) 
        
                   executor = ThreadPoolExecutor(max_workers=1) 
        
               return executor

If one of the bands fails, the whole tile production should fail, though. And Gunicorn workers should not affect Python processes, should they? 🤔

Sorry for all the guesswork. Are you sure your file is ok, @anup39? For example, try opening it in QGIS or so and zoom in and out.

If one of the bands fails, the whole tile production should fail, though.

Correct, so this could only happen if the data being read look OK but is faulty (corrupt overview, data being truncated while read), or through a race condition or cache pollution somewhere. Which would be almost impossible to debug without a reproducer...

	def create_executor() -> Executor:
	settings = get_settings()

	if not settings.USE_MULTIPROCESSING:
	return ThreadPoolExecutor(max_workers=1)

	executor: Executor

	try:
	# this fails on architectures without /dev/shm
	executor = ProcessPoolExecutor(max_workers=3)
	except OSError:
	# fall back to serial evaluation
	warnings.warn(
	"Multiprocessing is not available on this system. "
	"Falling back to serial execution."
	)
	executor = ThreadPoolExecutor(max_workers=1)

	return executor

In some data i get different color image tiles apart from true color .