Visualiser¶

In this notebook we'll explore how to finally visualise the result of an entire workflow (examples [1-5]) into a maps—static and interactive visualisers.

Let’s make an enriched layer pop!

Data source used:

PLUTO data from NYC Open Data. https://www.nyc.gov/content/planning/pages/resources/datasets/mappluto-pluto-change

Let’s make some pretty maps! 🗺️

In [1]:

Copied!

import urban_mapper as um

# Kick off UrbanMapper
mapper = um.UrbanMapper()
import urban_mapper as um

# Kick off UrbanMapper
mapper = um.UrbanMapper()

Preparing the Data and Layer¶

First, let’s load data, create a layer, and enrich it with average building floors.

Note that:

Loader example can be seen in examples/Basics/loader.ipynb
Urban Layer example can be seen in examples/Basics/urban_layer.ipynb
Enricher example can be see in examples/Basics/enricher.ipynb
Imputer example can be seen in examples/Basics/imputer.ipynb
Filter example can be seen in examples/Basics/filter.ipynb

In [2]:

Copied!





######
#
# We will on purpose do inline chaining of the methods to fasten the process.
#
######

# Load data
# Note: For the documentation interactive mode, we only query 5000 records from the dataset.  Feel free to remove for a more realistic analysis.
data = mapper.loader.from_huggingface("oscur/pluto", number_of_rows=5000, streaming=True).with_columns("longitude", "latitude").load()

# Create urban layer
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()

# Impute your data if they contain missing values
data = mapper.imputer.with_type("SimpleGeoImputer").on_columns("longitude", "latitude").transform(data, layer)

# Filter your data if they contain data beyond the bounding box your urban layer above
data = mapper.filter.with_type("BoundingBoxFilter").transform(data, layer)

# Map data to nearest layer
_, mapped_data = layer.map_nearest_layer(
    data,
    longitude_column="longitude",
    latitude_column="latitude",
    output_column="nearest_intersection",
)

# Enrich layer
enricher = (
    mapper.enricher
    .with_data(group_by="nearest_intersection", values_from="numfloors")
    .aggregate_by(method="mean", output_column="avg_floors")
    .build()
)

enriched_layer = enricher.enrich(mapped_data, layer)
######
#
# We will on purpose do inline chaining of the methods to fasten the process.
#
######

# Load data
# Note: For the documentation interactive mode, we only query 5000 records from the dataset.  Feel free to remove for a more realistic analysis.
data = mapper.loader.from_huggingface("oscur/pluto", number_of_rows=5000, streaming=True).with_columns("longitude", "latitude").load()

# Create urban layer
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()

# Impute your data if they contain missing values
data = mapper.imputer.with_type("SimpleGeoImputer").on_columns("longitude", "latitude").transform(data, layer)

# Filter your data if they contain data beyond the bounding box your urban layer above
data = mapper.filter.with_type("BoundingBoxFilter").transform(data, layer)

# Map data to nearest layer
_, mapped_data = layer.map_nearest_layer(
    data,
    longitude_column="longitude",
    latitude_column="latitude",
    output_column="nearest_intersection",
)

# Enrich layer
enricher = (
    mapper.enricher
    .with_data(group_by="nearest_intersection", values_from="numfloors")
    .aggregate_by(method="mean", output_column="avg_floors")
    .build()
)

enriched_layer = enricher.enrich(mapped_data, layer)

Static Visualisation¶

Now that we've cooked the various ingredients, let’s whip up a static map with Matplotlib to see our enriched layer.

In [3]:

Copied!





# Static visualisation
fig_static = (
    mapper
    .visual # From the visualiser module
    .with_type("Static")  # With type Static
    .show(columns=["avg_floors"])  # Show the avg_floors column
    .render(enriched_layer.get_layer()) # Render the enriched layer
)
# Static visualisation
fig_static = (
    mapper
    .visual # From the visualiser module
    .with_type("Static")  # With type Static
    .show(columns=["avg_floors"])  # Show the avg_floors column
    .render(enriched_layer.get_layer()) # Render the enriched layer
)

No description has been provided for this image

Interactive Visualisation¶

Now, let’s go fancy with an interactive Folium map—dark theme, because why not?

In [4]:

Copied!





# Interactive visualisation
fig_interactive = (
    mapper
    .visual # From the visualiser module
    .with_type("Interactive")  # With type Interactive
    .with_style({"tiles": "CartoDB dark_matter", "colorbar_text_color": "white"}) # Dark theme style
    .show(columns=["avg_floors"])  # Show the avg_floors column
    .render(enriched_layer.get_layer()) # Render the enriched layer
)
fig_interactive
# Interactive visualisation
fig_interactive = (
    mapper
    .visual # From the visualiser module
    .with_type("Interactive")  # With type Interactive
    .with_style({"tiles": "CartoDB dark_matter", "colorbar_text_color": "white"}) # Dark theme style
    .show(columns=["avg_floors"])  # Show the avg_floors column
    .render(enriched_layer.get_layer()) # Render the enriched layer
)
fig_interactive

Out[4]:

Make this Notebook Trusted to load map: File -> Trust Notebook

Be Able To Preview Your Visualiser¶

Want to check your visualiser’s setup? preview() shows you the type and style—perfect for shared projects!

In [5]:

Copied!

# Preview visualiser
print(mapper.visual.preview())
# Preview visualiser
print(mapper.visual.preview())

Visualiser: InteractiveVisualiser using Folium
Style: Default styling

None

Provide many different datasets to the same static visualization¶

You can load many datasets and feed the enricher with a dictionary. All the provided datasets should have the same columns provided in with_data, aggregate_by, etc.

The static visualizer looks into the enriched data with data_id column and uses it to show data with different markers.

In [6]:

Copied!





# Load CSV data
data1 = (
    mapper
    .loader
    .from_huggingface("oscur/pluto", number_of_rows=1000, streaming=True).with_columns("longitude", "latitude").load()
    # From the loader module, from the following file and with the `longitude` and `latitude`
)

# Load Parquet data
data2 = (
    mapper
    .loader
    .from_huggingface("oscur/taxisvis1M", number_of_rows=1000, streaming=True) # To update with your own path
    .with_columns("pickup_longitude", "pickup_latitude") # Inform your long and lat columns
    .with_map({"pickup_longitude": "longitude", "pickup_latitude": "latitude"}) ## Routines like layer.map_nearest_layer needs datasets with the same longitude_column and latitude_column    
    .load()
)

data = {
  "pluto_data": data1,
  "taxi_data": data2,
}

# Create a new urban layer to the data
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()

# Map datasets to the nearest layer
# Here the point is to say which intersection of the city maps with which record(s) in each of your datasets
# so that we can take into account when enriching.
_, mapped_data = layer.map_nearest_layer(
    data,
    longitude_column="longitude",
    latitude_column="latitude",
    output_column="nearest_intersection", # Will create this column in the data, so that we can re-use that throughout the enriching process below.
)

# Set up and apply enricher with debug enabled
enricher = (
    mapper
    .enricher # From the enricher module
    .with_data(
        group_by="nearest_intersection", values_from="numfloors", data_id="pluto_data"
    ) # Reading: With data grouped by the nearest intersection, and the values from the attribute numfloors
      #Both datasets should have the same group_by and values_from columns
    .aggregate_by(
        method="mean", output_column="avg_floors"
    ) # Reading: Aggregate by using the mean and output the computation into the avg_floors new attribute of the urban layer
    .with_debug()  # Enable debug to add DEBUG_avg_floors column which will contain the list of indices from the input data used for each enrichment
    .build()
)
enriched_layer = enricher.enrich(
    mapped_data, layer
)  # Data to use, Urban Layer to Enrich.

# Static visualisation
fig_static = (
    mapper
    .visual # From the visualiser module
    .with_type("Static")  # With type Static
    .show(columns=["avg_floors"])  # Show the avg_floors column
    .render(enriched_layer.get_layer()) # Render the enriched layer
)
# Load CSV data
data1 = (
    mapper
    .loader
    .from_huggingface("oscur/pluto", number_of_rows=1000, streaming=True).with_columns("longitude", "latitude").load()
    # From the loader module, from the following file and with the `longitude` and `latitude`
)

# Load Parquet data
data2 = (
    mapper
    .loader
    .from_huggingface("oscur/taxisvis1M", number_of_rows=1000, streaming=True) # To update with your own path
    .with_columns("pickup_longitude", "pickup_latitude") # Inform your long and lat columns
    .with_map({"pickup_longitude": "longitude", "pickup_latitude": "latitude"}) ## Routines like layer.map_nearest_layer needs datasets with the same longitude_column and latitude_column    
    .load()
)

data = {
  "pluto_data": data1,
  "taxi_data": data2,
}

# Create a new urban layer to the data
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()

# Map datasets to the nearest layer
# Here the point is to say which intersection of the city maps with which record(s) in each of your datasets
# so that we can take into account when enriching.
_, mapped_data = layer.map_nearest_layer(
    data,
    longitude_column="longitude",
    latitude_column="latitude",
    output_column="nearest_intersection", # Will create this column in the data, so that we can re-use that throughout the enriching process below.
)

# Set up and apply enricher with debug enabled
enricher = (
    mapper
    .enricher # From the enricher module
    .with_data(
        group_by="nearest_intersection", values_from="numfloors", data_id="pluto_data"
    ) # Reading: With data grouped by the nearest intersection, and the values from the attribute numfloors
      #Both datasets should have the same group_by and values_from columns
    .aggregate_by(
        method="mean", output_column="avg_floors"
    ) # Reading: Aggregate by using the mean and output the computation into the avg_floors new attribute of the urban layer
    .with_debug()  # Enable debug to add DEBUG_avg_floors column which will contain the list of indices from the input data used for each enrichment
    .build()
)
enriched_layer = enricher.enrich(
    mapped_data, layer
)  # Data to use, Urban Layer to Enrich.

# Static visualisation
fig_static = (
    mapper
    .visual # From the visualiser module
    .with_type("Static")  # With type Static
    .show(columns=["avg_floors"])  # Show the avg_floors column
    .render(enriched_layer.get_layer()) # Render the enriched layer
)

$No description has been provided for this image$

More visualiser primitives ?¶

Wants more? Come shout that out on https://github.com/VIDA-NYU/UrbanMapper/issues/9

Wrapping Up¶

Blimey, you’re ace! 🌟 You’ve made static and interactive maps like a pro. Tweak the styles or columns next—you’re unstoppable! 🚀