support in-memory raster

[giswqs]

Can localtileserver visualize an in-memory raster (e.g., a numpy array) with a known crs? This would be helpful for visualizing intermediate results without saving them to hard drive.

Rasterio in-memory files: https://rasterio.readthedocs.io/en/stable/topics/memory-files.html#memoryfile-bytesio-meets-namedtemporaryfile

[banesullivan]

Not presently possible, but wil be in a few weeks. I've been working on a new rasaterio-based source module for large-image (the backing library for localtileserver at this time). Once girder/large_image#1115 merges (FYI I haven't pushed most recent dev on this branch), this will be easily possible and I will follow up here.

@giswqs, would you please make a simple example using a numpy array with rasterio as an in-memory raster for me to experiment with?

[giswqs]

Here is an example

!wget https://github.com/opengeos/data/raw/main/raster/cog.tif

import rasterio
import numpy as np

# Open a raster dataset using rasterio
with rasterio.open('cog.tif') as src:
    # Read the raster data as a NumPy array
    data_array = src.read()
    width = src.width
    height = src.height
    count = src.count

    # Do something with the array
    new_array = data_array + 1000

    # Create an in-memory file using rasterio's MemoryFile class
    with rasterio.io.MemoryFile() as memfile:
        # Open the in-memory file as a GeoTIFF dataset with three bands
        with memfile.open(driver='GTiff', count=count, width=width, height=height, dtype=np.uint8) as dataset:
            # Write the data array to the dataset
            dataset.write(new_array)
            
             # It would be great if localtileserver can visualize the dataset without saving it to hard drive.
             # Save the dataset to a GeoTIFF file on disk. 
            with rasterio.open('new_array.tif', 'w', **src.profile) as dst:
                dst.write(dataset.read())

[banesullivan]

While localtilerserver now fully utilizes rasterio under the hood, getting this to work is a bit tricky and will require a full redesign of the internal REST API.

In brief, since localtileserver launches a flask-based tile server in a background thread we cannot share the in-memory object with that thread. To share the in-memory object, I'd have to drop the flask web backend and switch to tornado. I'll think about perhaps adding an additional tornado based web server for strictly tile endpoints to support this, but this "flavor" of localtileserver would not work in classic notebook, only JupyterLab

[giswqs]

Just figured out how to visualize in-memory raster dataset with localtileserver, making it possible to visualize rasterio/xarray datasets without writing files to hard drive. This is exciting!

opengeos/leafmap#650

import rasterio
from ipyleaflet import Map
from localtileserver import TileClient, get_leaflet_tile_layer

# Open a rasterio dataset
dataset = rasterio.open('https://open.gishub.org/data/raster/srtm90.tif')
data_array = dataset.read(1)

# Do some processing on the data array
data_array[data_array < 1000] = 0

# Create rasterio dataset in memory
memory_file = rasterio.MemoryFile()
raster_dataset = memory_file.open(driver='GTiff',
                                  height=data_array.shape[0],
                                  width=data_array.shape[1],
                                  count=1,
                                  dtype=str(data_array.dtype),
                                  crs=dataset.crs,
                                  transform=dataset.transform)

# Write data array values to the rasterio dataset
raster_dataset.write(data_array, 1)
raster_dataset.close()

# Read the dataset from memory
dataset_reader = rasterio.open(raster_dataset.name, mode='r')

# Create a TileClient and a Leaflet map
client = TileClient(dataset_reader)
t = get_leaflet_tile_layer(client, cmap='terrain', vmin=100, vmax=3000)
m = Map(center=client.center(), zoom=client.default_zoom)
m.add_layer(t)
m

[banesullivan]

Nice workaround! Thanks for following up here!

[giswqs]

I have implemented this in leafmap. It support both numpy.ndarray and xarray.DataArray. Closing this issue for now. Feel free to reopen if want to keep it for localtileserver's own implementation.
opengeos/leafmap#651

[banesullivan]

Reopening as a reminder for myself to look into this