Josh SolisJS Book an intro callBook a call

Construction tech · Computer vision

Blueprint Vision: Computer vision for construction takeoff and HVAC estimating

HVAC estimating starts with takeoff. Estimators count HVAC parts on construction drawings by hand. One building can take days. The symbols are small, dense, and different on each drawing set.

How it works

Find every symbol

A model draws a box around each symbol. Each box has a confidence score from 0 to 1.

A threshold decides which boxes stay. A high threshold removes wrong boxes. It also misses more real symbols.

Precision and recall move in opposite directions. The estimator must see both. My best detector reached mAP50 0.755.

Figure: a synthetic floor plan with detection boxes. Below it, a dot plot of confidence scores with a threshold line, and precision and recall curves that move in opposite directions as the threshold rises.

In the product

The estimator checks each AI suggestion

  1. The estimator clicks AI Assist. The model puts a box on each symbol that it finds.
  2. Each box shows a confidence score. A box with a low score can be wrong.
  3. The estimator rejects the wrong box. The app keeps this correction as training data.
  4. The estimator accepts the other boxes. The count changes from 12 to 15.

Simplified screen. The drawing, the scores and the counts are examples.

Animated figure: a takeoff app shows a mechanical drawing and a count of 12 supply diffusers. AI Assist adds four boxes with confidence scores. The estimator rejects the box with the score 0.41, because it is a thermostat. The app saves the correction as training data. The estimator accepts the other three boxes, and the count changes to 15.

45-second video

The correction flywheel

  1. A model makes a guess.
  2. A person corrects it.
  3. The fix becomes training data.
Transcript
  1. Estimators count HVAC parts on drawings by hand.
  2. The model draws a box on each part. Each box has a score.
  3. A threshold removes boxes with low scores.
  4. A person fixes the wrong boxes.
  5. Each fix becomes new training data.
  6. The model improves with each cycle.
  7. Data quality limited accuracy more than architecture.

The parts

  • A detection pipeline (YOLO, then RF-DETR) trained on labeled blueprint tiles.
  • An AI-assisted takeoff app: the estimator drives, AI suggests counts, duct sizes and symbol tags.
  • A correction flywheel: every human fix becomes new training data.
Show 2 more
  • Vector-text reading and tiled OCR to count grille tags from the PDF itself.
  • Revision tools: slip-sheeting, FFT phase-correlation auto-align, added/removed diff.

By the numbers

mAP50 0.755
Best detector
~1,100
Automated tests
Auto-aligned
Drawing revisions

What I learned

Data quality, not model architecture, limited accuracy. I changed the product to collect better data as people used it.