PERIA: Perceive, Interact, Reason
Building Tool-Augmented Visual Agents for Spatial Reasoning

NeurIPS 2026
Changye Li1, Meng Lu2, Yi Wu1, Ligeng Zhu3
1Institute for Interdisciplinary Information Sciences, Tsinghua University
2Virginia Tech    3NVIDIA

Abstract

While recent vision-language models (VLMs) demonstrate strong multimodal understanding, they remain limited in spatial reasoning tasks that require active evidence acquisition and multi-step visual interaction. This limitation suggests that relying solely on implicit visual representations from vision encoders is insufficient for recovering fine-grained spatial evidence. We introduce PERIA (PERception-Interaction-reason Agent), a tool-augmented visual agent for spatial reasoning tasks across map reasoning, visual probing, and vision reconstruction. PERIA uses two lightweight tool families: vision perception tools for exposing textual, symbolic, and spatial evidence, and vision interaction tools for manipulating visual context, tracing paths, and verifying spatial relations. To train PERIA, we develop a unified recipe that combines supervised tool-use trajectory synthesis, composite rewards, and Observation-Relaxed Group-in-Group Policy Optimization (OR-GIGPO) for effective multi-tool behavior. Experiments on 13 benchmarks from 8 datasets show that PERIA-8B improves over the Qwen3-8B backbone by 10.0% on in-distribution benchmarks and 4.4% on out-of-distribution benchmarks, while outperforming previous state-of-the-art baselines of similar size by 7.0%-14.8%. It also achieves performance comparable to much larger models such as Qwen3-VL-235B-A22B-Thinking and GPT-5, demonstrating the effectiveness of PERIA in enhancing spatial reasoning capabilities.

Key Highlights

  • 10.0% improvement on in-distribution spatial reasoning benchmarks over Qwen3-VL-8B-Thinking backbone
  • 4.4% improvement on out-of-distribution benchmarks
  • Outperforms previous SOTA baselines of similar size by 7.0%-14.8%
  • Achieves performance comparable to much larger models (235B parameters)
  • Novel OR-GIGPO training algorithm for multi-tool behavior learning
  • Lightweight tool families: Perception and Interaction tools

Overview

PERIA Overview
Figure 1: PERIA uses perception and interaction tools to actively acquire fine-grained spatial evidence for complex spatial reasoning tasks.

Method

🔍 Vision Perception Tools

Extract textual, symbolic, and spatial evidence from images:

  • OCR and text detection
  • Symbol recognition
  • Spatial relationship extraction

🎯 Vision Interaction Tools

Manipulate and explore visual context:

  • Path tracing
  • Visual context manipulation
  • Spatial relation verification
PERIA Results
Figure 2: PERIA-8B achieves substantial improvements across multiple spatial reasoning benchmarks, approaching the performance of much larger models.

Training Approach

PERIA employs a comprehensive training pipeline:

  1. Supervised Tool-Use Trajectory Synthesis: Generate high-quality demonstrations of tool usage
  2. Composite Rewards: Multi-faceted reward signals for tool selection and reasoning
  3. OR-GIGPO (Observation-Relaxed Group-in-Group Policy Optimization): Novel RL algorithm for effective multi-tool behavior learning

Citation

If you find PERIA useful for your research, please cite our paper:

@inproceedings{li2026peria, title={Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning}, author={Li, Changye and Lu, Meng and Wu, Yi and Zhu, Ligeng}, booktitle={Advances in Neural Information Processing Systems (NeurIPS)}, year={2026} }

Acknowledgment

We thank the NeurIPS 2026 reviewers for their valuable feedback. This work was supported by Tsinghua University and NVIDIA.