Foundational Knowledge Base Perception & Decision-Making for Autonomous Vehicles at Road Signals A structured study reference covering 9 core topics for thesis preparation How to use this document Each topic follows the same structure: (1) what it is in plain terms, (2) why it matters for your pipeline, (3) the fundamental technique(s) underneath it, (4) better/alternative solutions available today, and (5) how it connects to the other topics. Read Topics 1–6 first (perception layer), then 7–9 (decision layer) — the pipeline section at the end shows how they all snap together. 0. The Big Picture — How These 9 Topics Fit Together Your supervisor gave you two groups of topics. They are not 9 unrelated things — they are two layers of one pipeline: Layer A — Perception (“seeing” the road) • OpenCV, YOLO, DETR, Grounding DINO, DINO — tools/models that detect what is on the road • Obstacle detection — the applied task these models perform • Distance calculation, Speed calculation — turning detections into numbers (how far, how fast) Layer B — Decision (“acting” on the road) • Path planning — the global route (map-level) • Motion planning — the real-time reaction (frame-level), directly consuming Layer A’s output In one sentence: cameras + detectors (Layer A) tell the car what is around it and how far/fast; motion planning (Layer B) uses that to decide brake, turn, or continue, while path planning keeps the overall route sensible. TOPIC 1 OpenCV A toolkit, not a model — the glue underneath everything else What it is (plain terms) OpenCV is a free, open-source library of ready-made image-processing functions — not an AI model. Think of it as a toolbox: resize an image, convert colors, detect edges, calibrate a camera, draw boxes on a frame. It has been the default computer-vision library for 20+ years. Why it matters here

Every AV vision pipeline uses OpenCV somewhere — reading video frames, preprocessing images before feeding them to YOLO/DETR, converting pixel coordinates to real-world coordinates, or doing classical lane detection (edge detection + Hough transform) as a lightweight baseline. Fundamental technique underneath • Edge detection (Canny), color filtering (HSV thresholding), and geometric transforms are the classical building blocks it provides. • Camera calibration functions (intrinsic/extrinsic matrices) are essential for later distance calculation. Better / alternative solutions available OpenCV itself has no real competitor as a general-purpose CV library — the real choice is whether to rely on its classical algorithms (fast, interpretable, but fragile in bad lighting/weather) versus deep-learning models for the actual detection task. For a thesis, OpenCV is rarely the contribution — it is infrastructure. Your novel work happens in Topics 2–5 and 7–9. Foundation to nail down OpenCV = library for image I/O, preprocessing, and classical CV. It is not a competitor to YOLO/DETR — it usually works alongside them. TOPIC 2 YOLO (You Only Look Once) The fast, real-time object detector What it is (plain terms) YOLO looks at an entire image once and, in a single pass, predicts all bounding boxes and class labels simultaneously — hence “you only look once,” as opposed to older methods that scanned the image in multiple stages. This single-pass design is what makes it fast enough for real-time video. Why it matters here YOLO is the default choice for detecting vehicles, pedestrians, traffic lights, and signs in live video because it can run at high frame rates — critical when a car needs to react within milliseconds. Fundamental technique underneath • Single-stage detection: one convolutional network predicts boxes + classes directly, no separate “region proposal” step. • Anchor boxes or anchor-free grids to handle objects of different sizes. • Non-Maximum Suppression (NMS) to remove duplicate overlapping boxes for the same object. Better / alternative solutions available The YOLO family itself keeps improving — recent versions such as YOLO26 and YOLOv12 remain strong real- time choices, and RTMDet wins on pure throughput at over 300 FPS. But the more important alternative is

architectural: RT-DETR (see Topic 3) has been shown to beat YOLO on both speed and accuracy simultaneously, which used to be YOLO’s exclusive advantage. For custom domain fine-tuning specifically, RF- DETR is now reported as the first real-time detector to pass 60 mAP on the COCO benchmark, ahead of pure YOLO variants. Model Strength Trade-off YOLO26 / YOLOv12 Fast, CPU/edge-friendly, huge community & tooling Slightly behind newer transformer models on small/occluded objects RTMDet Highest raw throughput (300+ FPS) Optimized for speed over top accuracy RT-DETR / RF-DETR Beats YOLO on speed AND accuracy in benchmarks Newer, smaller community, more compute to train TOPIC 3 DETR (Detection Transformer) Detection re-imagined using transformers What it is (plain terms) DETR throws away hand-designed steps like anchor boxes and NMS. Instead, it treats detection as a direct set- prediction problem: a transformer looks at the whole image and directly outputs a fixed set of “this is an object, here is its box and class” predictions, matched one-to-one with ground truth during training. Why it matters here The original DETR was accurate but too slow for real-time driving. That changed with RT-DETR, which was shown to outperform state-of-the-art YOLO detectors and other DETR variants in both speed and accuracy — removing DETR’s biggest historical weakness for AV use. Fundamental technique underneath • Self-attention across the whole image (global context, unlike YOLO’s more local convolutional view). • Set prediction with bipartite matching — no NMS post-processing needed. • RT-DETR variants skip NMS entirely at inference, simplifying and speeding up the pipeline. Better / alternative solutions available Within the DETR family, DINO-DETR (see Topic 5) is repeatedly cited as a top accuracy baseline; CO-DETR and RF-DETR push accuracy further, with RF-DETR using a DINOv2 backbone for strong domain adaptability. On a real autonomous-driving dataset, RT-DETR achieved an F1-score of 0.672 at IoU=0.5, clearly ahead of YOLOv8s (0.458) and YOLO-NAS (0.526) — a strong, citable data point for your literature review. Foundation to nail down DETR = transformer-based, global-context, NMS-free detection. RT-DETR is the version that made this real-time-viable and is the fair comparison point against YOLO in your thesis.

TOPIC 4 Grounding DINO Detect anything you can describe in words What it is (plain terms) Grounding DINO is a transformer-based model that combines vision and language: instead of only recognizing a fixed list of trained categories, it lets you type a text description (e.g. “a pedestrian crossing the road”) and it finds matching objects in the image, even if that exact label was never in its training data. Why it matters here Standard detectors (YOLO/DETR) can only find what they were trained to find. Road-signal detection can involve many sign types across regions; Grounding DINO's zero-shot property means you can prompt for a new category without retraining — useful when your dataset does not cover every sign or scenario. Fundamental technique underneath • Cross-modal alignment: image features and text features are learned in a shared space so a caption can “point to” a region. • Zero-shot detection: no fine-tuning needed for new object categories described in text. Better / alternative solutions available YOLO-World offers similar open-vocabulary, text-prompted detection with a lighter YOLO-based backbone, trading some flexibility for speed. SAM 3 extends the same zero-shot idea to pixel-level segmentation rather than just boxes. The practical caveat for your thesis: zero-shot accuracy trails fine-tuned models, so production-grade systems typically graduate from prompting to a properly trained model once enough labeled data exists — meaning Grounding DINO is better positioned as a bootstrapping/data-labeling tool than your final deployed detector. Foundation to nail down Grounding DINO's core value is flexibility (new classes without retraining), not top accuracy. Decide early whether your thesis needs that flexibility or whether a fixed, fine-tuned detector (YOLO/RT-DETR) is enough — this is a real design choice to justify in your proposal. TOPIC 5 DINO Careful: two different things share this name What it is (plain terms) — and the naming trap This is the single easiest point to embarrass yourself on in a defense if unclear, so lock it in:

• Grounding DINO (Topic 4) = a full detection system that finds objects from text prompts. • DINO / DINO-DETR (this topic) = (a) a self-supervised pretraining method for building strong general-purpose image features without labels, and (b) the name of a specific high-accuracy detection head architecture (“DINO- DETR”) that adopts advanced query formulation combined with contrastive denoising training. Why it matters here DINO-style pretrained backbones (e.g., DINOv2) provide deep, globally-aware image features that downstream detectors can build on. In fact, RF-DETR (mentioned in Topics 2–3) specifically uses a DINOv2 backbone to extract these rich features, then adds a lightweight detection head — which is exactly why RF-DETR adapts so well to new domains with less data. Fundamental technique underneath • Self-supervised learning: the model learns useful visual features from unlabeled images (no manual annotation needed for pretraining). • DINO-DETR adds contrastive denoising training on top of the DAB-DETR query formulation to sharpen detection accuracy. Better / alternative solutions available As a backbone choice, DINOv2 is currently considered best-in-class for domain adaptability across specialized domains like aerial imagery, medical scans, and industrial inspection — road-scene imagery falls into this “specialized domain” category well. As a detection head, DINO-DETR remains a strong accuracy baseline, though it is not optimized for real-time speed the way RT-DETR/RF-DETR are. Foundation to nail down If your supervisor says “DINO,” clarify whether they mean the text-prompted Grounding DINO detector or the self- supervised DINO backbone/DINO-DETR head — they solve different problems. TOPIC 6 Obstacle Detection The applied task that Topics 1–5 exist to perform What it is (plain terms) Obstacle detection is not a separate algorithm — it is the applied use of an object detector (YOLO, RT-DETR, Grounding DINO, etc.) specifically to find things a vehicle must not collide with: other vehicles, pedestrians, cyclists, road debris, and static infrastructure. Why it matters here This is the direct input to your decision layer. Everything in Topics 2–5 is about choosing which model produces this output most reliably and quickly. Fundamental technique underneath

• Run a chosen detector every frame (or every few frames) → get bounding boxes + class labels + confidence scores. • Object tracking (e.g. ByteTrack, DeepSORT) is typically layered on top so the same obstacle is recognized across consecutive frames rather than re-detected from scratch — this consistency is what later enables speed calculation (Topic 8). Better / alternative solutions available The main axis of improvement here is robustness under difficult conditions. One domain-generalization study found all tested models suffered significant performance drops under challenging environmental conditions such as heavy snowfall and low-light scenarios, and stressed the need for geographically diverse training data and domain adaptation techniques — directly relevant if your dataset needs to reflect Bangladeshi road/weather conditions rather than only Western datasets like KITTI or BDD100K. TOPIC 7 Distance Calculation Turning a 2D box into a real-world number What it is (plain terms) Once you have detected an object, distance calculation answers “how far away is it, in meters?” This is essential for deciding when to brake, how much time is available to react, and whether a gap is safe to cross. Why it matters here Distance feeds directly into motion planning (Topic 9): a detection alone tells you what and roughly where in the image; distance tells you how urgent the reaction needs to be. Fundamental technique underneath Three broad families exist, in increasing cost/accuracy: • Sensor-based (LiDAR/Radar): highest precision, but expensive, which is why cost is the main barrier to mass- market adoption. • Monocular vision (single camera): low-cost, using the pinhole camera model — known real-world object size (e.g. license plate dimensions) plus its size in pixels gives distance via simple geometry. Cheap but suffers from scale ambiguity and sensitivity to environmental disturbances. • Binocular/stereo vision (two cameras): triangulates distance from the disparity between two viewpoints, more accurate than monocular but roughly doubles compute and hardware cost. Better / alternative solutions available Plain monocular geometric estimation is known to be noisy: on standard driving datasets, average monocular ranging error can reach around 20% and binocular error around 12%, which is a real safety concern for planning and decision-making. Newer approaches reduce this considerably:

• Deep-learning monocular depth estimation networks learn depth directly from single images, moving beyond simple geometry. • Fiducial-marker methods (e.g., using standardized license-plate character height as a reference) have been shown to reduce the standard deviation of distance estimates by 35% compared to plate-width-based methods, giving smoother, more reliable readings. • Sensor-fusion approaches combining monocular and binocular vision with adaptive filtering (e.g., Unscented Kalman Filter fusion) have reduced average relative error by over 40% compared to either method alone. Approach Cost Typical accuracy note LiDAR / Radar High Most precise, but costly for mass- market cars Monocular (geometric) Lowest ~20% average error on standard datasets; scale- ambiguous Monocular (deep learning depth) Low– medium Better than geometric alone, still camera- only Binocular / stereo Medium ~12% average error; needs two calibrated cameras Sensor fusion (mono+stereo, filtered) Medium 40%+ error reduction vs. single- method baselines Thesis angle worth considering “Improving monocular distance estimation accuracy for low-cost AV systems” is a well-supported, fundable thesis narrative on its own — it directly matches your stated goal of producing better output than prior work, and monocular-only keeps hardware costs realistic for a university project.

TOPIC 8 Speed Calculation How fast is that obstacle moving? What it is (plain terms) Speed calculation estimates how quickly a detected object is moving (and in what direction), usually relative to your own vehicle. Distance alone tells you how far something is right now; speed tells you how quickly that distance is closing. Why it matters here Distance + speed together give time-to-collision (TTC) — a single number that is often the actual trigger condition for braking or evasive decisions in motion planning. Fundamental technique underneath • Track the same object across consecutive frames (via object tracking, building on Topic 6). • Measure how its estimated distance (Topic 7) changes over the known time interval between frames. • Convert that pixel/metric displacement over time into a velocity estimate, ideally smoothed with a filter (e.g. Kalman filter) to reduce frame-to-frame noise. Better / alternative solutions available Because speed calculation is derived from distance calculation, it inherits and amplifies the same errors — noisy distance estimates produce noisy speed estimates. This is why the sensor-fusion and filtering approaches mentioned in Topic 7 (e.g., adaptive Kalman filtering across monocular and stereo estimates) matter twice as much here: smoothing is what prevents erratic estimates from triggering unnecessary braking or acceleration. TOPIC 9 Path Planning vs. Motion Planning Global route vs. real-time reaction — the most commonly confused pair What they are (plain terms) • Path planning: the global question — computing an optimal trajectory from point A to point B, considering safety, efficiency, and real-time adaptability, while respecting road constraints such as lanes, intersections, and speed limits, as well as dynamic obstacles and traffic rules like signals and stop signs. • Motion planning: the local, moment-to-moment question — generating detailed trajectories based on the current vehicle state and behavior decision outputs, ensuring the trajectory stays smooth and comfortable while respecting dynamic constraints such as speed and acceleration. Why it matters here

Motion planning is explicitly described as serving as the bridge between upstream perception modules and downstream low-level control signals — this is precisely the role your obstacle detection + distance/speed calculation output is meant to feed into. Path planning, by contrast, operates more at the map/route level and interacts with things like traffic-signal timing at a higher level. Fundamental techniques underneath Path planning • Optimization-based methods such as Mixed-Integer Programming (MIP) and variational calculus are widely used because they can jointly encode road constraints, dynamic obstacles, and traffic rules while finding a globally optimal route. • At intersections specifically, reinforcement-learning approaches (e.g. Deep Q-Networks) have been used to help vehicles navigate safely and efficiently, and newer frameworks combine hierarchical reinforcement learning with graph neural networks so vehicles and traffic signals coordinate dynamically. Motion planning • Two core sub-functions: trajectory generation (the actual path shape/speed profile) and tactical decision-making (what maneuver to perform — stop, yield, change lane). • “Hybrid” methods that combine data-driven (learning-based) and logic-driven (rule-based/analytic) techniques are an active research direction, aiming to get both the adaptability of learning and the predictability of rules. Better / alternative solutions available The clearest emerging direction across recent surveys is a shift toward transformer-based and foundation-model planners, alongside uncertainty-aware planning that explicitly models how confident the system is in its own predictions — relevant given that your perception layer's distance/speed estimates always carry some error. For a thesis, a realistic and well-scoped contribution is a hybrid motion-planning module: use rule-based logic for safety-critical constraints (never violate a red light, always maintain minimum following distance) combined with a learned model for smoothness/efficiency — rather than attempting a full end-to-end learned planner, which is a much larger undertaking. Foundation to nail down If your supervisor asks you to justify a design choice, the safest one-line answer is: “Path planning decides where the car should generally go; motion planning decides what it should do right now given what the perception layer just detected.” Putting It All Together — A Coherent Thesis Narrative Based on everything above, here is one way these 9 topics combine into a single, defensible thesis pipeline aimed at your stated goal (judging road signals, making efficient driving decisions, improving on existing models): • Step 1 — Detect: Choose a detector (YOLO26 for speed, or RT-DETR/RF-DETR for accuracy) and fine-tune it on a road-signal-heavy dataset, optionally using OpenCV for preprocessing and Grounding DINO to help bootstrap labels for classes you lack data for. • Step 2 — Estimate: Compute distance (start with monocular geometric or a fiducial-marker method for low cost; consider fusion/filtering for better accuracy) and derive speed via tracking across frames.

• Step 3 — Decide: Feed detections + distance + speed into a motion-planning module that outputs a driving action (accelerate / brake / turn / stop) in real time, using a hybrid rule-based-plus-learned approach. • Step 4 — Improve & Prove: Benchmark your pipeline against a baseline (e.g., plain YOLOv8s) on a recognized dataset or your own road-signal dataset, and show quantitative improvement — this is your “better than previous work” contribution.

Quick-Reference Glossary (for your defense) Term One-line definition mAP Mean Average Precision — the standard accuracy metric for object detectors FPS Frames per second — the standard real-time speed metric NMS Non-Maximum Suppression — removes duplicate overlapping detection boxes IoU Intersection over Union — measures how well a predicted box overlaps the true box Zero-shot detection Detecting object categories the model was never explicitly trained on Backbone The feature-extraction part of a network that other components (like a detection head) build on Time-to-collision (TTC) Distance ÷ closing speed — how long until impact if nothing changes Self-supervised learning Learning useful features from unlabeled data (used to pretrain backbones like DINOv2) Next step suggestion Once you and your supervisor settle on which layer(s) this thesis will actually contribute to (perception model comparison, distance/speed estimation, or the decision/motion-planning module), a focused literature-review reading list for that specific sub-area will be far more useful than trying to read deeply on all 9 topics at once.

Flipso