Reading a traffic sign is deceptively hard for an automated driving system. A camera sees color, glyphs, and shape in fine detail, but its estimate of where the sign actually sits in the world is weak, and it degrades in glare, rain, fog, and low light. Radar is the mirror image: it measures range and motion robustly through weather, but it carries little of the visual texture needed to tell a speed-limit plate from a yield triangle. A grant issued to Waymo LLC on July 7, 2026, US12674866B2, is directed to combining the two so that each sensor covers the other's blind spot when the system reads a sign.

The conventional pattern is late fusion: run a camera detector, run a radar detector, then try to reconcile two lists of candidate objects downstream. The disclosed technique moves the merge earlier. As the record states, the system obtains a set of perspective camera images and a set of radar images of the environment, generates camera features with a first neural network and radar features with a second neural network, and then processes both feature sets together to identify the signs. The signals are combined while they are still learned feature maps, not after each modality has already committed to a decision.

The disclosed systems and techniques facilitate efficient detection and classification of traffic signs in driving environments. The disclosed techniques include, obtaining, using a sensing system of a vehicle a first set of perspective camera images of an environment and a second set of radar images of the environment. The techniques further include generating, using a first neural network, one or more camera features characterizing the first set of images, generating, using a second neural network, one or more radar features characterizing the second set of images, and processing the one or more camera features and the one or more radar features to obtain an identification of one or more traffic signs in the environment.— Detection and classification of traffic signs using camera-radar fusion, US12674866B2

How the fusion is structured

The mechanism the independent claim describes turns on a shared coordinate system. Each camera feature is generated for a respective pixel of a coordinate system and for a respective time; each radar feature is generated for that same pixel and that same time. Because both modalities are indexed to the same grid and the same clock, their features line up cell by cell. The claim then calls for fusing the camera features with the radar features into a fused tensor that aggregates them both across pixels of the coordinate system and across multiple times. In other words, the fused representation is spatial and temporal at once: it stacks what each location looked like to the camera and to the radar over a window of frames.

Getting the two sensors onto a common grid is itself part of the disclosure. The record describes mapping camera features from a perspective coordinate system to a ground-surface coordinate system, so the image-plane view and the radar returns can be expressed in the same ground-plane frame before they are combined. That alignment is what makes per-pixel fusion meaningful rather than a coincidental overlay of two unrelated projections.

A third neural network then processes the fused tensor. Per the claim, its output comprises the semantic content of the signs, and the record elaborates that the third network is built as a backbone feeding classification heads. The semantic content is not merely "a sign is present": it includes the sign type, the sign's value, its relevance to the vehicle, and the sign's location. Distinguishing relevance matters in practice, because a sign facing a cross street or a parallel lane may be legible yet not binding on the ego vehicle. The final step in the claim closes the loop to actuation: a driving control system controls the vehicle based on that semantic content.

Refinements and where it sits in the field

The disclosure adds several refinements around the core pipeline. A fourth neural network can produce an auxiliary sign identifier that is fed back into the third network to sharpen the reading. The three primary networks are trained together rather than as isolated stages, which lets the fusion representation and the sign classifier co-adapt. The record also describes identifying and eliminating duplicate signs, so a single physical sign observed across multiple frames or by both sensors resolves to one entity rather than several.

Placed against the broader state of perception engineering, the approach reflects a wider industry shift from hand-tuned late fusion toward learned early fusion on a common spatial representation, and it treats time as a first-class axis alongside space. The same design vocabulary recurs across the assignee's other issued work. A companion grant on high-throughput point-cloud processing, US12675887B2, applies a temporal neural network with time-point queries to LiDAR; a diffusion-model trajectory-prediction grant, US12668282B2, forecasts multi-agent futures; and a sensor grant, US12671793B2, adjusts a sensor's field-of-view volume when degradation is detected. On the planning side, related grants address collision-cost-based safe distancing, US12668277B1, and speed reasoning under uncertainty when pulling over near occluded parking spots, US12669341B2.

The sign-reading grant is classified under CPC groups spanning radar signal processing and autonomous vehicle control, including G01S 7/417, G01S 13/867, and B60W 60/001. Taken on its own terms, the patent is directed to a specific engineering choice: read the sign from a fused, ground-referenced camera-radar tensor, and let the network that reads it also be the thing that feeds control, so the robustness of radar and the discrimination of a camera arrive at the classifier together rather than being reconciled after each has already guessed.