Semantic vs Instance Segmentation on Aerial Imagery
Semantic segmentation gives every pixel of an image a class: this pixel is building, that one is road, the next is tree. Instance segmentation goes one step further and separates the objects of a class from each other: this pixel is building number 12, that one is building number 13.
The difference in one line
Semantic answers "how much". Instance answers "how many, and which one".
| Semantic | Instance | |
|---|---|---|
| Output | One mask per class | One polygon per object |
| Two touching houses | One mask | Two objects |
| Can you count with it | No | Yes |
| Typical model | U-Net, DeepLab | Mask R-CNN, Segment Anything |
What each one gives you on a map
Semantic is a land cover map with sharp edges: a building mask, a road mask, a vegetation mask. Polygonise it and a whole block of terraces is one feature. Right for area statistics, impervious surface, canopy cover, change between two dates. Wrong for anything that needs a count.
Instance is an inventory. Every roof, tree crown or pool arrives as its own polygon with its own row in the attribute table. You can count, measure each one, join a height to each one. The cost is that the model has to decide where one object ends, which is hard on a terraced row, and the failures look like it: two houses merged, one crown split in three.
Which one the QGIS tools produce
- Semantic: Deepness and most of the models in its zoo, the Semi-Automatic Classification Plugin, Mapflow's land cover model.
- Instance: the Segment Anything family, so Geo-SAM, samgeo and the SAM plugins for QGIS. One object per click.
- AI Segmentation, the plugin we make at TerraLab for QGIS, is an instance tool. You name the object, a building, a tree, a pool, or draw one example of it, and it returns one polygon per object found in the zone. The building footprint guide shows what that looks like on a dense block.
, is an instance tool. You name the object, a building, a tree, a pool, or draw one example of it, and it returns one polygon per object found in the zone. The Questions people ask
Is object detection the same as instance segmentation?
No. Object detection returns a rectangle around each object. Instance segmentation returns its outline, pixel by pixel. For a map you almost always want the outline.
Can I turn a semantic mask into instances?
Sometimes. Polygonise the mask, then split the polygons where they neck. It works on separated things like tree crowns in an orchard. It fails on a terrace, where there is no neck to cut at.
Does the resolution of the imagery change the answer?
Yes. At 10 m a house is two pixels and there is nothing to separate, so satellite land cover is semantic by necessity. At 20 cm a house is thousands of pixels and instance segmentation becomes possible.
The QGIS AI hub collects what AI can and cannot do inside QGIS.
